=Paper=
{{Paper
|id=Vol-3696/ELEARNING_paper_14
|storemode=property
|title=Defining Quality of Experience in e-Learning: a Literature Review
|pdfUrl=https://ceur-ws.org/Vol-3696/article_14.pdf
|volume=Vol-3696
|authors=Athanasia Kylafi,Nikolaos Misirlis,Yiannis Nikolaidis
|dblpUrl=https://dblp.org/rec/conf/elearning/KylafiMN23
}}
==Defining Quality of Experience in e-Learning: a Literature Review==
https://ceur-ws.org/Vol-3696/article_14.pdf
Defining Quality of Experience in e-Learning: a Literature Review
Athanasia Kylafi1, Nikolaos Misirlis2 and Yiannis Nikolaidis1
1
University of Macedonia, 156 Egnatia Street, 546 36, Thessaloniki, Greece
2
HAN University of Applied Sciences, R31, 6824DG, Arnhem, The Netherlands
Abstract
Our research aims to investigate the “quality of experience” in education and particularly in
the context of e-learning. Firstly, we study separately the concepts of “quality” and
“experience”, as well as the underlying meaning of these terms. Secondly, through a scoping
review among papers of the last decade, we study how “quality of experience”, as a whole, is
defined in e-learning. Our purpose is to set a solid basis for the clarification of “quality of
experience” in e-learning, considering that it is important to have a clear view of concepts
before proceeding with a research related to them. As a result of our review, we reach useful
answers to questions such as “What is quality of experience in e-learning?”, “Is quality of
experience different from user experience?” gaining deeper understanding of the examined
concepts.
Keywords 1
quality, quality of experience, e-learning, literature review, scoping review
1. Introduction
There is no doubt that e-learning, which is defined as “the learning supported by digital electronic
tools and media” [1], especially after the Covid pandemic, has become a necessity in the educational
sector globally, either it concerns school education, higher education or vocational education and
training. Thus, it is reasonable that there is an increasing interest in optimizing e-learning procedures
and tools in order to offer e-learning services of high quality. This interest derives not only from the
educational organizations that wish to pioneer and to be competitive, but also from researchers and
academics that constantly look for new ways to evaluate and improve e-learning environments.
Therefore, it comes that the concept of quality is crucial when it relates to education and especially e-
learning, mainly because the latter is delivered via digital technologies, which in fact mediate the
educational process and transform the connections among the participants (teachers and students).
In general, it is difficult to define quality in a unique way, because it is a subjective, relational and
multidimensional concept. Actually, there is a long debate about its definition. Ehlers [2]
distinguishes three components of this debate on quality, as shown in fig. 1. According to him, it is
not easy to define quality because of a) different interpretations of quality, b) different players with
different perspectives of quality, c) different forms of quality.
However, generally speaking and according to Juran, quality can be defined as “the degree to
which a product / service meets or exceeds expectations / requirements of customers” and “fitness for
use”. According to Crosby, quality means “conformance to requirements”. According to ISO 8402
[3], quality is “the characteristics of a product or service that bear on its ability to satisfy stated or
implied needs”. Finally, another definition of quality is given in the Qualinet White Paper [4]:
“Quality is the outcome of an individual‟s comparison and judgment process. It includes perception,
reflection about the perception, and the description of the outcome. In contrast to definitions which
Proceedings for the 14th International Conference on e-Learning 2023, September 28-29, 2023, Belgrade, Serbia
EMAIL: nancykylafi@uom.edu.gr (A. 1); nikolaos.misirlis@han.nl (A. 2); nikolai@uom.edu.gr (A. 3)
ORCID: 0009-0004-2249-0618 (A. 1); 0000-0001-8815-2081 (A. 2); 0000-0003-4201-7786 (A. 3)
© 2023 Copyright for this paper by its authors.
Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
CEUR Workshop Proceedings (CEUR-WS.org)
CEUR
ceur-ws.org
Workshop ISSN 1613-0073
Proceedings
126
see quality as a „set of inherent characteristics‟, quality is considered in terms of the evaluated
excellence or goodness, of the degree of need fulfillment, and in terms of a „quality event‟”.
Figure 1: Dimensions of Quality [2]
As can be seen from the above, quality may refer either to services or to products and according to
this basic distinction it could be categorized into further types, e.g. “quality of conventional services”
and “quality of e-services”. Following a conceptual path from quality to the central subject of our
study, namely the quality of experience in e-learning, we present in what follows the state of the art
knowledge regarding the related concepts.
Specifically about services, Ehlers [2] mentions Donabedian‟s statement of the relational nature of
quality: “quality is the degree of conformance between a performed service and the goals set for this
service”. Similarly, according to Parasuraman et al. [5], service quality can be defined as “perceptions
(that) result from a comparison of consumer expectations with actual service performance”. From this
point of view, service quality is measured by calculating the difference between expectations and
perceptions.
With the advent of the internet to most aspects of human activity in modern societies, more and
more conventional services are extended to electronic, namely e-services. The latter are web-based
services, which are delivered through electronic media (e.g. computers, smartphones, tablets). Thus,
the characteristics of services and e-services vary to a great extent and, therefore, the quality
dimensions of services and e-services vary too [6]. One well-known definition of the quality of e-
services is the following: “E-service quality is defined as the extent to which a website facilitates
efficient and effective shopping, purchasing, and delivery” [7]. Of course, this definition does not
actually reflect the wide range of e-services that exist nowadays (e.g. e-learning, e-banking, e-
government, etc.), but we evaluate positively the fact that it focuses on two important elements,
efficiency and effectiveness. According to the above definition, e-service quality is the degree of
efficiency and effectiveness of services delivered to users on the internet.
Taking one step further, e-learning - as an educational service offered through digital resources on
the internet - is undeniably a form of e-service. Quality of e-learning is defined as “the philosophy of
achieving excellence in all aspects of education through e-learning technologies” [8].
Exploiting the previously described definition of quality, we consider the quality of e-learning as
the degree to which an e-learning course meets or exceeds the expectations / requirements of learners,
providing them with educational experiences. Quality of e-learning may refer to the educational
technology, the educational content, the teaching-learning process, the student assessment, the
administrative support etc.
Through e-learning environments, the participants acquire various educational experiences. An
experience is defined as “an individual‟s stream of perception and interpretation of one or multiple
events” [4]. Experiences in e-learning may refer to teachers, to students or both. Moreover,
127
experiences are connected to the quality that characterizes every e-learning environment (and every
educational environment generally), meaning that there may be good or bad experiences. Ιn other
words, there may be high or low quality of experience. The latter is defined as follows:
“Quality of Experience (QoE) is the degree of delight or annoyance of the user of an application or
service. It results from the fulfillment of his or her expectations with respect to the utility and/or
enjoyment of the application or service in the light of the user‟s personality and current state” [4].
Discussing quality of experience in the context of e-learning, quality is an important factor for the
future of e-learning [2]. On the other hand, experience of the learners is a crucial factor for the overall
success of an e-learning course, because it directly affects their learning outcome(s). Through their
experience, the learners fulfill their expectations and gain satisfaction and knowledge. So, from the
learners‟ perspective, the concept of quality is perceived as their experience, based on their
expectations and satisfaction.
The purpose of this paper is to explore the literature of the last decade, in order to determine and
clarify the concept of QoE in e-learning. We also aim to gain deeper understanding of the concept, by
searching for answers to research questions such as:
Research Question 1: What is QoE in e-learning?
and
Research Question 2: Is QoE different from user experience?
To serve our purpose we conduct a scoping literature review, as this type of review is the most
appropriate to gather knowledge, clarify concepts and identify gaps on a particular topic. The findings
of this review hopefully provide a clear understanding of QoE to researchers and other stakeholders in
the field of e-learning (e.g. academics, practitioners, organizations). They also indicate the recent
trends of research on QoE in e-learning.
The paper is organized as follows. In Section 2 we present the methodology of the scoping
literature review. In Section 3 we go on with the results of our review and in Section 4 we discuss the
findings. In Section 5 we provide our conclusions.
2. Literature review methodology
A literature review is a form of research for knowledge synthesis. There are many literature review
types, such as narrative review, systematic review, scoping review, mapping review etc. [9, 10]. For
the purpose of this paper, we consider the scoping review as the most appropriate. Scoping reviews
belong to the category of rapid reviews [9]. A scoping review is suitable for exploring, identifying,
mapping, reporting or discussing characteristics or concepts across a breadth of evidence sources [11].
It is used for clarifying concepts, identifying knowledge gaps and providing an overview of the
existing knowledge. The present study is guided by the PRISMA 2020 statement [12] and the
PRISMA extension for Scoping Reviews Checklist [10, 13], and was conducted in 5 stages [9]:
Stage 1: identifying the research question(s)
Stage 2: identifying relevant publications / studies
Stage 3: selection of publications / studies
Stage 4: charting the data
Stage 5: collating, summarizing and reporting the results
The purpose and the research questions of our research (Stage 1) are mentioned at the end of
Section 1. Next, (as per Stage 2) we selected the database of Scopus and determined the search
criteria to trace the relevant literature: boolean search of the terms “quality of experience” AND “e-
learning” in title-abstract-keywords, filtered with the year range “2014-2023”. We continued with the
inclusion or exclusion of the publications that had arisen (Stage 3). Publications that either were
irrelevant to our research or did not provide any definition of QoE were excluded. We then studied the
selected publications and extracted the useful information (Stage 4). Finally we summarized the
findings (Stage 5).
Our search in Scopus initially brought 122 records as a result. After the removal of 4 records due
to duplication, retraction and other reasons, 118 records remained. At the first screening 67 records
were excluded as irrelevant. We proceeded to the second screening for retrieval of the reports related
128
to our purpose, where 45 reports were retrieved. We assessed thoroughly all 45 retrieved reports for
eligibility. 29 of them were excluded with reasons (no definitions detected). We finally included 16
studies in the review.
3. Results
In Table 1 we provide the results of our review in chronological order. For each reviewed study we
present the definitions/meanings of QoE mentioned in it. In addition, we present the research topics in
the field of e-learning that the study relates to. The related research topics are of great interest because
they offer more detail on the meaning of QoE. Moreover, we note the type of publication.
Table 1
Definitions of QoE in e-learning and related research topics
Author(s) & Type of Related research Definition/meaning of "quality of experience"
Year publication topics in "e-learning"
Comsa, Molnar, journal video-based online QoE is perceived by learners experiencing heterogeneous
Tal, Imhof, article learning, video traffic (which means with different traffic loads) video services in
Bergamin, prioritization with e-Learning.
Muntean, Machine Learning
Muntean,
Trestian
2023 [14]
Chakkaravarthy, journal web service QoE is a user-perceived metric representing service
Kumar article recommendation system satisfaction.
for course selection
2023 [15]
Souchet, Philippe, journal virtual reality, serious QoE is the subjective experience by users, based on usual
Lévêque, Ober, article games for learning variables considered in VR such as Presence and Flow.
Leroy
2022 [16]
Thang, conference online learning, video QoE is the extent users are satisfied with contents provided
Watanobe, Kiran, paper streaming by applications or services.
Paik
2022 [17]
Bieg, Schatz, conference virtual reality, digital QoE is one of the six measures for experience, along with
Egger-Lampl, paper training hedonic user experience, pragmatic user experience, VR-
Roszipal, Kinzer induced symptoms, sense of presence, overall satisfaction.
2022 [18]
Ali, Simba conference video streaming QoE is the overall acceptability of an application or service,
paper as perceived subjectively by the end-user.
2021 [19]
Gao journal internet-based resources, QoE is the user’s point of view of the whole performance of
article English learning a system. QoE is the degree to which users of an application
2021 [20] or service are happy or annoyed. It is based on the user’s
personality, current state, and expectations for the utility
and/or enjoyment of the application service. QoE ultimately
leads to the acceptability of the system, application, or
service. QoE is the construct concerning how well a system
provides functions for a user to complete particular tasks.
129
Doumanis, conference hybrid Virtual Reality, QoE is a measure of how users experience a hybrid VR
Economou, paper 360-degree video application.
Argyriou streaming
2021 [21]
Pinto, Monteiro, conference Virtual Reality, QoE is related to the evaluation of each participant towards
Melo, Cabral, paper gamification, second the application.
Bessa language learning
2021 [22]
Pal, Vanijja, Patra conference online learning, QoE is defined as “the overall acceptability of an application
paper videoconferencing or service, as perceived subjectively by the end user”. From
2020 [23] applications, multimedia the definition itself it is clear that QoE is a superset of QoS
quality that includes all possible complete end-to-end system
effects, along with the user expectations.
Idrizi, Filiposka, conference online education, Qualities of Experience are the measurements of how
Trajkovik paper learning styles students felt and were satisfied during the online classes.
2018 [24]
Kong, Liu, An conference Virtual Reality, virtual
QoE is “the overall acceptability of an application or service,
paper training system as perceived subjectively by an end-user”. “QoE is the
2018 [25] degree of delight or annoyance of the user of an application
or service. It results from the fulfillment of his or her
expectations with respect to the utility and/or enjoyment of
the application or service in the light of the user’s
personality and current state”. QoE is a multidimensional
quality that can be decomposed in a set of perceptual
attributes called features.
Davcev, conference mobile learning, QoE is used as an overall acceptability of an application or
Jakimovski, paper multimedia content, service, as perceived subjectively by the end-user and
Scepanovic mobile cloud computing represents a multidimensional subjective concept that is not
easy to evaluate.
2018 [26]
Moldovan, conference mobile learning, QoE is a main factor contributing to user’s engagement with
Ghergulescu, paper multimedia multimedia services.
Muntean
2017 [27]
Fernandes, conference MOOC QoE is used to describe how students evaluate a service.
Cardoso, paper QoE "is how a user perceives the usability of a service when
Marcelino in use". QoE is "the degree to which a system meets the
tacit and explicit expectations of the user for the
2015 [28] experience".
Karadimce, conference mobile learning, QoE is a metric that quantifies the multifaceted,
Davcev paper multimedia content, multidimensional factors that influence the perceived
collaborative cloud quality. QoE is defined as the degree of delight or
2015 [29] service annoyance of a person whose experience involves an
application, service, or system. QoE is no longer an
expression for user satisfaction of using a service, but it is a
degree of delight or annoyance, which is a more dynamic
measure of personal experience.
4. Discussion of the findings
4.1. Basic characteristics of the studies
10 articles out of 16 (62.5%) are published between 2020 and 2023 and the rest (37.5%) from 2015
to 2019. The majority of them (75%) are conference papers, which implies that the particular research
area is still unexplored, thus very promising.
130
The research topics of selected studies vary depending on the e-learning area, the applied
technologies and the type of educational content. Topics that refer to the field or a sub-field of e-
learning are the following:
video-based online learning [14]
online learning [17, 23, 24]
digital training [18]
language learning [20, 22]
virtual training [25]
mobile learning [26, 27, 29]
mooc [28]
Topics that refer to the technology or the content of e-learning are:
video streaming [14, 17, 19, 21]
machine learning [14]
web service for course selection [15]
virtual reality [16, 18, 21, 22, 25]
gamification [16, 22]
internet-based resources [20]
videoconferencing [23]
multimedia [23, 26, 27, 29]
learning styles [24]
cloud computing [26, 29]
Finally, another characteristic is that all 16 studies approach e-learning from the learners‟
perspective.
4.2. Ηow is QoE defined in e-learning?
4.2.1. What is QoE in e-learning?
According to the definitions presented in Table 1, QoE is considered as:
a measure of satisfaction [15, 17, 24]
a measure of experience [16, 18, 21, 29]
the acceptability of an application or service [19, 20, 23, 25, 26]
the delight or annoyance from an application or service [20, 25, 29]
a subjective perception by the user [14, 15, 16, 20, 22, 26, 28]
a multidimensional concept [25, 26, 29]
a factor of user's engagement [27]
From the above conceptual variety, we confirm that QoE in e-learning is multidimensional and
subjective, thus it is defined depending on the dimensions of the research interest. However, very
often the definitions overlap and have many elements in common. We infer that experience and
satisfaction are related concepts, both user-centric and user-perceived, and their measurement
determines QoE. We also infer that most of the studies adopt mainly two definitions of QoE, either
directly or with some paraphrase, which are a) the definition by the ITU (i.e., “the overall
acceptability of an application or service, as perceived subjectively by an end-user”) and b) the
definition by the Qualinet White Paper (i.e., “the degree of delight or annoyance of the user of an
application or service”). At this point it is noteworthy to mention the statement of Karadimce and
Davcev [29] that “QoE is no longer an expression for user satisfaction of using a service, but it is a
degree of delight or annoyance, which is a more dynamic measure of personal experience”. This
indicates the subjective character of QoE.
According to the above definitions, QoE is affected by factors such as:
the type of device and the quality of the Internet connectivity [14],
presence, flow and symptoms in VR [16, 21]
quality variation, stalling, and initial delay in a streaming session [17]
131
user expectations and context [19]
user‟s personality, current state, and expectations for the utility and/or enjoyment of the
application service [20]
Additionally, as there is no single definition of QoE, it is difficult to evaluate the particular
concept [26] or, at least, there is no single measure of it. The measures of QoE vary as dependable on
the various factors that are taken into account in every research.
4.2.2. Is QoE different from user experience?
The answer to this question is that QoE and user experience (UX) are very similar concepts, since
both of them refer to users‟ experience with digital technologies. However, they are not exactly the
same. We infer that UX is the experience perceived by the user, while QoE is a measure for UX [16,
18, 21, 29]. More specifically, Bieg et al. [18] mention an interesting description of QoE, “as one of
the six measures for experience, along with hedonic user experience, pragmatic user experience, VR-
induced symptoms, sense of presence, overall satisfaction”. From this statement we conclude that
QoE and UX are not the same.
5. Conclusions
Concluding our review, we find out that there is not a specific, widely accepted definition of QoE
in e-learning. The adopted definitions in the field of e-learning are also used in other fields, as QoE is
defined for all e-services. Moreover, most studies in this review approach e-learning from a technical
view, mainly through computer science, thus QoE is mainly related with technical aspects of the e-
learning systems. QoE is a concept often used in a technical context [30]. We agree with Kist and
Brodie [30], that the learning experience differs significantly from the general consumer experience.
We also agree with Ehlers and Hilera [31], according to whom “quality in the field of e-learning is an
especially diverse field, because it brings together the field of education, technology, and economy in
order to contribute a) to societal development, b) to innovate formal, non formal, and informal
learning opportunities, and c) empower learners as citizens to take part in our emerging learning and
information societies”. We believe that QoE in e-learning is a concept that refers to learning
processes, through learning environments, where learning objectives exist and learning outcomes
happen. Learners are more than consumers of a service, they acquire knowledge and they develop
skills and attitudes in a long-term horizon, apart from their short-term experiences. So, QoE in e-
learning needs to be defined more specifically and this is certainly a research gap which should be
studied in the near future.
This review also reveals that the research focus concerns only the learners‟ perspective. This was
expected, as learners are the main end-users in e-learning and actually the user-perceived experience
refers to the learners‟ experience. However, this fact can be seen as a research gap, because teachers
are highly involved in e-learning too (except self-learning cases), and their experience may have a
significance for the overall QoE, e.g. in higher education. This review also shows that QoE is not easy
to evaluate and this can be considered as another research gap too.
This review has some limitations regarding the generalization of the findings. Due to the fact that
it was conducted in only one database, Scopus, more research is needed, expanded in more databases
in order to come to general results. The time span is also a limitation of this review.
6. References
[1] K. S. Basak, M. Wotto, P. Bélanger, E-learning, M-learning and D-learning: Conceptual
definition and comparative analysis, E-Learning and Digital Media 15 (4) (2018) p. 191-216.
https://doi.org/10.1177/2042753018785180
132
[2] U.-D. Ehlers, Quality in e-learning: the learner as a key quality assurance category, Vocational
Training European Journal no. 29 (2003) p. 3-15.
https://www.cedefop.europa.eu/files/etv/Upload/Information_resources/Bookshop/349/29_en_Eh
lers.pdf
[3] ISO 8402, Quality - Vocabulary, International Organisation for Standardisation, Geneva, 1986.
[4] K. Brunnström, S. A. Beker, K. de Moor, A. Dooms, S. Egger et al., Qualinet White Paper on
Definitions of Quality of Experience, Output from the 5th Qualinet meeting (Novi Sad) (March
2013). https://hal.science/hal-00977812
[5] A. Parasuraman, V. A. Zeithaml, L. L. Berry, A Conceptual Model of Service Quality and Its
Implications for Future Research, Journal of Marketing 49 (4) (1985) p. 41-50.
https://doi.org/10.1177/002224298504900403
[6] S. Firdous, R. Farooqi, Service Quality To E-Service Quality: A Paradigm Shift, in: Proceedings
of the 9th International Conference on Industrial Engineering and Operations Management
(Bangkok, Thailand), IEOM Society International, 2019, p. 1656-1666.
http://www.ieomsociety.org/ieom2019/papers/404.pdf
[7] V. A. Zeithaml, A. Parasuraman, A. Malhotra, A conceptual framework for understanding e-
service quality: implications for future research and managerial practice, Working Paper, Report
No. 00-115, MSI, Cambridge, MA, USA (2000). https://thearf-org-unified-
admin.s3.amazonaws.com/MSI/2020/06/MSI_WP_00-115.pdf
[8] M. R. M. Veeramanickam, P. Ramesh, Analysis on quality of learning in e-Learning platforms,
Advances in Engineering Software 172 no. 103168 (2022), ISSN 0965-9978.
https://doi.org/10.1016/j.advengsoft.2022.103168
[9] H. Arksey, L. O'Malley, Scoping studies: towards a methodological framework, International
Journal of Social Research Methodology 8 (1) (2005) p. 19-32.
https://doi.org/10.1080/1364557032000119616
[10] F. Campbell, A. C. Tricco, Z. Munn et al., Mapping reviews, scoping reviews, and evidence and
gap maps (EGMs): the same but different - the “Big Picture” review family, Systematic Reviews
12 no. 45 (2023). https://doi.org/10.1186/s13643-023-02178-5
[11] M. D. J. Peters, C. Marnie, H. Colquhoun et al., Scoping reviews: reinforcing and advancing the
methodology and application, Systematic Reviews 10 no. 263 (2021).
https://doi.org/10.1186/s13643-021-01821-3
[12] M. J. Page, J. E. McKenzie, P. M. Bossuyt et al., The PRISMA 2020 statement: an updated
guideline for reporting systematic reviews, Systematic Reviews 10 no. 89 (2021).
https://doi.org/10.1186/s13643-021-01626-4
[13] A. C. Tricco, E. Lillie, W. Zarin, K. K. O'Brien, H. Colquhoun, D. Levac, D. Moher, M. D. J.
Peters, T. Horsley, L. Weeks, S. Hempel et al., PRISMA extension for scoping reviews
(PRISMA-ScR): checklist and explanation, Annals of Internal Medicine 169 (7) (2018) p. 467-
473. https://doi.org/10.7326/M18-0850
[14] I.-S. Comsa, A. Molnar, I. Tal, C. Imhof, P. Bergamin, G.-M. Muntean, C. H. Muntean, R.
Trestian, Improved Quality of Online Education Using Prioritized Multi-Agent Reinforcement
Learning for Video Traffic Scheduling, IEEE Transactions on Broadcasting 69 (2) (2023) p. 436-
454. https://doi.org/10.1109/TBC.2023.3246815
[15] M. Chakkaravarthy, G. S. Kumar, A Quality of Experience-based Recommender System for E-
learning Resources, International Journal on Recent and Innovation Trends in Computing and
Communication 11 (5s) (2023). https://doi.org/10.17762/ijritcc.v11i5s.6590
[16] A. D. Souchet, S. Philippe, A. Lévêque, F. Ober, L. Leroy, Short- and long-term learning of job
interview with a serious game in virtual reality: influence of eyestrain, stereoscopy, and
apparatus, Virtual Reality 26 (2) (2022) p. 583-600. https://doi.org/10.1007/s10055-021-00548-9
[17] T. C. Thang, Y. Watanobe, R. U. Kiran and I. Paik, Towards QoE Management for Post-
Pandemic Online Learning: Invited Paper, 14th International Conference on Knowledge and
Systems Engineering (KSE) (Nha Trang, Vietnam), IEEE, 2022.
https://doi.org/10.1109/KSE56063.2022.9953769
[18] T. Bieg, R. Schatz, S. Egger-Lampl, B. Roszipal, K. Kinzer, Better Experience, Better
Performance? Results of a Study on VR Training Effectiveness in Healthcare, 14th
133
International Conference on Quality of Multimedia Experience (QoMEX) (Lippstadt, Germany),
IEEE, 2022. https://doi.org/10.1109/QoMEX55416.2022.9900889
[19] M. K. Ali, F. Simba, Performance evaluation of 3G (UMTS Network) for E-Learning Video
Streaming, IEEE AFRICON (Arusha, Tanzania), IEEE, 2021.
https://doi.org/10.1109/AFRICON51333.2021.9570866
[20] H. L. Gao, The Impact of Quality of Experience of Chinese College Students on Internet-Based
Resources English Learning, Future Internet 13 (7) no. 162 (2021).
https://doi.org/10.3390/fi13070162
[21] I. Doumanis, D. Economou, L. Argyriou, Measuring and Comparing QoE of Hybrid VR
Applications under Increased Network Load, 7th International Conference of the Immersive
Learning Research Network (iLRN) (Eureka, CA, USA), IEEE, 2021.
https://doi.org/10.23919/iLRN52045.2021.9459316
[22] R. D. Pinto, P. Monteiro, M. Melo, L. Cabral, M. Bessa, Does gamification in virtual reality
improve second language learning?, International Conference on Graphics and Interaction (ICGI)
(Porto, Portugal), IEEE, 2021. https://doi.org/10.1109/ICGI54032.2021.9655286
[23] D. Pal, V. Vanijja, S. Patra, Online Learning During COVID-19: Students‟ Perception of
Multimedia Quality, 11th International Conference on Advances in Information Technology
(IAIT) (Bangkok, Thailand), ACM, 2020. https://doi.org/10.1145/3406601.3406632
[24] E. Idrizi, S. Filiposka, V. Trajkovik, Character Traits in Online Education: Case Study, in: S.
Kalajdziski, N. Ackovska (Eds.) ICT Innovations 2018, Engineering and Life Sciences, ICT
2018, Communications in Computer and Information Science, 940, Springer, Cham, 2018, p.
247-258. https://doi.org/10.1007/978-3-030-00825-3_21
[25] X. Kong, Y. Liu, M. An, Study on the Quality of Experience Evaluation Metrics for Astronaut
Virtual Training System, in: J. Chen, G. Fragomeni (Eds.) Virtual, Augmented and Mixed
Reality: Interaction, Navigation, Visualization, Embodiment, and Simulation, VAMR 2018,
Lecture Notes in Computer Science, 10909, Springer, Cham, 2018, p. 416-426.
https://doi.org/10.1007/978-3-319-91581-4_31
[26] D. Davcev, G. Jakimovski, S. Scepanovic, Model of M-Learning by Multimedia Content
Delivery from mCloud to Mobile Devices, in: M. Auer, D. Guralnick, I. Simonics (Eds.)
Teaching and Learning in a Digital World, ICL 2017, Advances in Intelligent Systems and
Computing, 715, Springer, Cham, 2018, p. 795-802. https://doi.org/10.1007/978-3-319-73210-
7_92
[27] A.-N. Moldovan, I. Ghergulescu, C. H. Muntean, Analysis of Learner Interest, QoE and EEG-
Based Affective States in Multimedia Mobile Learning, 17th International Conference on
Advanced Learning Technologies (ICALT) (Timisoara, Romania), IEEE, 2017, p. 398-402.
https://doi.org/10.1109/ICALT.2017.93
[28] A. F. Fernandes, J. Cardoso, M. J. Marcelino, A Systematic Mapping Applied to MOOC‟s Study,
in: Proceedings of the 7th International Conference on Computer Supported Education (CSEDU)
(Lisbon, Portugal), Vol. 1, SciTePress, 2015, p. 444-449.
https://doi.org/10.5220/0005483904440449
[29] A. Karadimce, D. Davcev, Collaborative cloud service model for delivering multimedia content
in mCloud, 10th IEEE International Conference on Collaborative Computing: Networking,
Applications and Worksharing (CollaborateCom 2014) (Miami, USA), IEEE, 2015, p. 469-474.
https://doi.org/10.4108/icst.collaboratecom.2014.257484
[30] A. A. Kist, L. Brodie, Quality of Service, Quality of Experience and Online Learning, 2012
Frontiers in Education Conference Proceedings (Seattle, WA, USA), IEEE, 2012.
https://doi.org/10.1109/FIE.2012.6462223
[31] U.-D., Ehlers, J. R. Hilera, Special Issue on quality in e-learning, Journal of Computer Assisted
Learning, 28 (1) (2012) p. 1-3. https://doi.org/10.1111/j.1365-2729.2011.00448.x
134