E-learning is essential in some instances and, with the right approach, can produce a high level of knowledge. With the growing demand for e-learning, the problem of online education quality assessment is becoming increasingly important. The paper singles out the advantages and disadvantages of online education for students, education institutions and employers, and analyses approaches to online education quality assessment in institutions of higher education. This assessment should be carried out based on a scientific methodology using modern tools of economic and mathematical modeling and information technology. The authors suggest that the quality of online education shall be evaluated from the standpoint of the project approach, with due regard to commercial, reputation, and strategic aspects, i. e. regarding an online education program (or a set of programs) as an investment project and assessing the efficiency of the given project for its core participants - the higher education institution and its students. The obtained performance indicators will constitute the objective assessment of the quality of e-learning in a particular higher education institution. These indicators are the response of the socio-economic system to the quality of online education. To estimate the values of the weight coefficients that define the importance of the opinions of target groups, the authors propose using a scheme based on Fishburn sequences. They also note that it is feasible to design a software application for assessing the quality of online education.
A major challenge faced by most of the world’s economies is how to improve the quality of management and the quality of produced goods and services to boost their competitiveness in the global market. However, it is impossible to meet the challenge without generating a higher return from education across the board and especially in the * field of higher education. Knowledge is the catalyst for the development of a modern economy and the main factor of production.
One of the forms of training is e-learning, which is now becoming more and more
popular. According to P. Ginns and R.A. Ellis, “e-learning is a growing and important
part of student experiences of learning at a university internationally” [
E-learning is characterized by both positive and negative sides. Recently, the quality
of online education has been the subject of much debate. Nevertheless, e-learning is
essential in some instances (for example, during COVID-19 or for physically
challenged people) and, with the right approach, can produce a high level of knowledge (for
example, a second university degree for employed and highly motivated people). “As
the recent coronavirus outbreak prompted universities to start shifting classes online
either for a few weeks or for the remainder of the spring semester of 2020, e-learning
and remote education have popped up as the magical alternative for in-person classes
in the time of the COVID-19 pandemic” [
It should be noted that nowadays specialists are facing the need for lifelong learning, which is due to the high dynamism of the professional environment. To keep their competitive edge in the labor market, modern specialists with a degree have constantly to enhance and expand their knowledge, skills, and abilities. And, in the long run, this trend is likely to intensify further, while e-learning is the best choice for working specialists.
With the growing demand for e-learning, the problem of online education quality assessment is becoming increasingly important. This assessment should be carried out based on a scientific methodology using modern tools of economic and mathematical modeling and information technology since information technology has become a serious decision support tool.
A significant number of scientific works are devoted to the problem of assessing the
quality of online education. For example, different approaches to identifying criteria
for online education quality assessment can be found in the works of the following
authors: D. Al-Fraihat, M. Joy, R. Masa'deh and J. Sinclair [
Despite a significant number of works on the problems of assessing the quality of online education, this research area still requires attention. We believe that there is a need to create a methodology for assessing the quality of online education, with objective and unbiased evaluation.
The objective of the paper is to analyze approaches to online education quality assessment in higher school and to propose an original approach to address the issue, which should form the basis for creating a software application for online education quality assessment in higher school.
The Advantages and Disadvantages of Online Education Currently, institutions of higher education are introducing e-learning to expand their markets, reduce costs, and achieve more flexibility. E-learning provides its participants with several undeniable benefits, but it is not free of some essential drawbacks given in Table 1.
Parties Advantages Disadvantages Stu- Time-saving (for example, for transpor- Difficulty in choosing a training program; dents tation); cost-saving (transport and other learning materials may be of low quality; expenditures); the possibility to combine during e-learning, difficulties may arise with work and learning; the accessibility of understanding the material, especially it's a learning materials 24/7; the availability practical part; students’ low motivation can of education for physically challenged be a constraining factor for obtaining the necpeople; the possibility of individualiza- essary knowledge, skills, and abilities; insuftion of the trajectory and content of ficient level of social communication skills. learning.
Edu- The possibility to expand the market; The problem of motivating students to comcation broad market coverage; reduction of in- plete the full cycle of training; the problem of insti- frastructure maintenance costs; the pos- checking the level of knowledge, skills, and tutions sibility to collaborate with other educa- abilities of students; the problem of confirmtion establishments, including foreign ing the identity of students during learning ones. and checking the acquired knowledge; high labor intensity of development of training courses.
The possibility of employee training on Lack of a clear understanding of the presence the job. and level of graduates’ competencies. Employers a Designed by the authors partly using the materials [5; 8]. 3
Analyzing Approaches to Online Education Quality
Assessment in Higher School
“E-Learning (EL) is explained as a network affinity group sharing their information,
knowledge, proficiency, and conferring education to many learners geographically in
the same or diversified. EL is learning and teaching online and sharing resources
electronically” [
Online education quality assessment is a complex task; this is due to its qualitative
nature, which, in some instances, may lead to a certain degree of subjectivity. The
problem of assessing the quality of e-learning is complex and subjective. The success of this
task will be determined by the comprehensiveness of the approach and the correct
choice of criteria and assessment methods.
- institutional factor (institutional affairs, administrative affairs, research,
reputation);
- technological factor (development and sustainability of technological
infrastructure, the functionality of technological infrastructure,
accessibility, reusability, interface design);
- instructional design factor (clarifying expectations, personalization,
selecting proper learning scenarios, organizing learning resources,
currency, and accuracy of learning resources);
- pedagogical factor (student-centredness, communication, and
interactivity, social aspects, learning environments, assessment, learning
resources);
- student support (administrative support, technical support);
-teachers support (technical assistance in course development,
administrative support, pedagogical support);
- evaluation factor (cost-effectiveness, learning effectiveness, student
satisfaction, teachers’ satisfaction) [
As M.G. Sergeeva notes, the quality of education including e-learning “... does not
have a generally accepted formalized description and a particular set of criteria that
characterize its essence” [
Evaluation of the Quality of E-Learning Using the ProjectBased Approach The authors believe that in a higher education institution the quality of e-learning as a whole or a particular online education program can be evaluated using the project-based approach, i. e. regarding an online education program (or a set of programs) as an investment project and assessing the efficiency of the given project for its core participants – the higher education institution and its students. In this case, we are talking about calculating the commercial efficiency of the project, which is taking into account the benefits and costs of its core participants.
The obtained performance indicators will constitute the objective assessment of the quality of e-learning in a particular higher education institution. These indicators are the response of the socio-economic system to the quality of online education. See Figure 1.
In the classic sense, the efficiency of an investment project is a category that
expresses “... the compliance of the results and costs of the project with the goals and
interests of its participants including, if necessary, the state and the population” [
In the authors’ opinion, when assessing the commercial efficiency of an online
education project for its core participants, it is feasible to apply the concept of evaluating
the commercial efficiency of investment projects in the real economy, proposed in the
monograph by M.A. Bakumenko [
In the case of assessing the commercial efficiency of an online education project for its participants: the commercial component (СС) shows how a project participant achieves their financial goals during the project’s design period; the strategic component (SC) shows how an online education project affects the strategic development of a project participant; the reputation component (RC) reflects a change in the reputation of a project participant, resulted from participation in the online education project. See Figure 2.
Methodology for Assessing the Quality of Online Education The essence of the methodology for assessing the quality of online education, proposed by the authors, is briefly described in Table 3.
Following the proposed approach, it is possible to talk about high-quality e-learning in a higher education institution subject to the conditions as follows: 1. ≥ 0, i 1, 2 . I. e. the commercial component for all online education project participants should be non-negative. 2. The online education project will have a positive impact on the process of strategic development of the higher education institution and the student. 3. R C i 0 , i 1, 2 . I. e. the online education project will have a positive impact on the reputation of the higher education institution and the reputation of the student (graduate) as a specialist in a particular subject area.
The reputation component of the proposed methodology (for a particular online education project participant) can be determined by equation (1), subject to the constraints (2).
T R C rt rt wt ,
t1 T wt 1 , ≥ 0, t 1, T .
t1 Where RC is a change in the image (reputation) of the OE project participant, resulting from participation in the OE project; rt is the response of t target group to the OE project participant after project implementation; rt is the response of t target group to the OE project participant before project implementation; w t is a weight coefficient that defines the importance of the opinion of t target group for the OE project participant; T is the number of target groups.
The values of rt and rt are in the range of [– 1; 1] and correspond to a special scale that numerically expresses the attitude of a certain target group towards the OE project participant. For this purpose, one can apply the scale presented in [14; 15]. To define the values of rt and rt , one can use expert technologies combined with qualitative research methods (media monitoring, survey, questionnaire, business conversation, etc.).
If RC > 0, the reputation of the OE project participant has improved after participation in the project; if RC < 0, the reputation of the OE project participant has deteriorated after participation in the project; if RC 0, the reputation of the OE project participant has remained the same. (1) (2) Indica- Online education project participant tor Higher education institution E-learner Com- CC1 is the value of NPV, reflecting the CC2 is the expected value of NPV, remer- commercial efficiency of a particular OE flecting the commercial efficiency of a cial program (or a set of programs) over the particular OE program for some hypocom- project’s design period for the higher ed- thetical (average) student. ponent ucation institution. The project’s design period equals the (СС) The project’s design period equals the du- sum of the duration of learning through a ration of learning through a particular OE particular OE program and five years afprogram (if several education programs terward (related to the possibility of putare included in the analysis, the longest ting the obtained knowledge, skills, and training interval is taken as the design pe- abilities into practice as well as to the relriod). evance of this knowledge).
We are talking about efficiency calcula- To find the expected value of NPV, the tions in the operating phase of the project scenario analysis method should be aplife cycle. The information base for calcu- plied. Based on the processing of statistilations is the data provided by the finan- cal information and using expert technolcial service of the higher education insti- ogies, the cash flows of three scenarios tution. The process of calculations takes are predicted: optimistic, basic, and pesinto account the benefits and costs of the simistic ones, and the probability of ochigher education institution, determined currence is determined for each scenario. by a particular OE program (or their to- The process of calculations takes into actal). count the benefits and costs of the student The discount rate is determined by the over the project’s design period, due to available alternative investment opportu- participation in a particular OE program nities following the modern methodology (including opportunity cost). for assessing the efficiency of investment The discount rate is determined by the projects. available alternative investment opportunities.
Strate- SC1 characterizes the degree to which the SC2 characterizes the possibility of the
gic OE project encourages the implementa- graduate to advance in the long run. It is
com- tion of potential projects in the future defined by experts based on the available
ponent (the technique presented in the work [
Reputa- RC1 characterizes a change in the reputa- RC2 characterizes a change in the reputation comt-ion of the higher education institution, tion of the worker (specialist), resulted ponent resulted from participation in the OE from participation in the OE project. (RC) project. b Formed by the authors partly using the materials [14; 16].
The reputation of a higher education institution (as well as the reputation of a specialist) is one of the most significant intangible assets and strongly influences the process of their strategic development. To preserve (improve) reputation, it is very important to adhere to ethical principles in one’s activities. An organization’s ethical behavior will enhance its competitiveness (Table 3).
The paper [
To estimate the values of the weight coefficients that define the importance of the opinions of target groups, the authors propose using a scheme based on Fishburn sequences. 6
Fishburn Sequence-Based Scheme to Estimate the Values of
Weight Coefficients
Notably, in equation (1), it is crucial correctly to estimate the unknown values of w t
components of the weight vector. The search for these estimates can be based on
various methods and models, including the concept of applying statistical games combined
with antagonistic games, described in detail in the monograph by A.V. Sigal [
Let us take a closer look at the methodology for estimating the values of w t weight
coefficients, based on Fishburn sequences [
Without loss of generality of reasoning, we can assume that the target groups under
consideration are arranged in descending order in terms of the importance of their
opinions (as seen by a certain OE project participant). This indicates that the values of the
weight coefficients generate the non-ascending sequence: w 1 w 2 … w T. Thus, on
the components of the weight vector, one or another linear order relation is set by the
subjective preferences of a certain OE project participant. Linear order relations were
studied in detail by Peter C. Fishburn and are given, for example, in the monograph by
R. I. Trukhaev [
A simple linear order relation refers to the relations w 1 w 2 … w T. A partially strengthened linear order relation refers to the relations wt w1 ... wt1 , t 2, T .
Let us set the weight vector W (w 1; w 2;…; w T). The vector’s w t components shall satisfy all the constraints (2). What is more, as the target groups under consideration are arranged in descending order in terms of the importance of their opinions, the values of the weight coefficients satisfy the inequalities w 1 w 2 … w T.
If in the subjective view of a certain OE project participant, a simple linear order
relation holds for the unknown values of the weight coefficients, Peter C. Fishburn
suggests that the estimates of w t components generate an arithmetic progression, and if a
partially strengthened linear order relation holds, then they form a monotonic geometric
progression (see for example [
We call generalized Fishburn progressions the progressions {w 1; w 2;…; w T}, satisfying all the constraints (2): generalized arithmetic Fishburn progressions are arithmetic progressions that satisfy all the constraints (2), generalized geometric Fishburn progressions are geometric progressions that satisfy all the constraints (2).
It can be easily proved that generalized arithmetic Fishburn progressions represent arithmetic progressions of the form wt 1 T 1 x t 1 x 2 T T 2 t 1 x , t 1, T ,
T 2 2 T whose difference satisfies the inequality x
2 T T 1 , while generalized geometric Fishburn progressions represent geometric progressions of the form wt x 1 x t1 1 x
x t1 , t 1, T ,
x T 1 1 x T
whose common ratio satisfies the inequality x 0. For example, the basic properties
of generalized Fishburn progressions are given in the monograph [
Monotonic sequences {w 1; w 2;…; w T} that satisfy all the constraints (2) shall be regarded as Fishburn sequences.
We can estimate the unknown values of w t weight coefficients according to the following three-step scheme based on Fishburn sequences.
Step 1. To define what type of linear order relations the unknown values of w t weight coefficients shall satisfy.
Step 2. To choose a convenient monotonic sequence {a 1; a 2;…; a T} of nonnegative numbers, the sum of which is a positive number, and the sequence itself satisfies the type of linear order relations that the unknown values of w t weight coefficients shall satisfy. Such a sequence {a 1; a 2;…; a T} can be referred to as a sequence generating a Fishburn sequence.
Step 3. To estimate the unknown values of w t weight coefficients by the elements of the corresponding Fishburn sequence by the equation
a t wt T , t 1, T .
a j j1
Notably, as a sequence generating the corresponding Fishburn sequence, one can
choose monotonic progressions of natural numbers, including
{a 1; a 2;…; a T} {1; 1;…; 1} or {a 1; a 2;…; a T} {T; T – 1;…; 1}, as well as such
sequences of natural numbers as Fibonacci numbers, Euclid numbers, Mersenne numbers,
Fermat numbers (see for example [
Having estimated the unknown values of w t weight coefficients by the elements of the selected Fishburn sequence, one should calculate the desired change in the image (reputation) of the OE project participant, resulted from participation in the OE project, applying the elements of the selected Fishburn sequence to the equation (1) to calculate the RC indicator. 7
Concerning the Creation of a Software Application for Online Education Quality Assessment in Higher School At present, information technology has become an integral part of management and managerial decision-making. According to the authors, the proposed methodology for assessing the quality of online education in higher schools should form the basis of the corresponding software application, since all the calculations within the proposed methodology are rather time-consuming.
Note that the commercial component of the proposed methodology can be found for
each online education project participant using special software designed to assess the
effectiveness of investment projects and to manage them. Examples of such software
solutions well established in the market are Primavera and Microsoft Project, a
comparative analysis of which is given, in particular, in the paper [
However, the proposed methodology has two more important specific components: the strategic component and the reputation one. The presence of these components in the proposed methodology necessitates the development of an appropriate software application that would facilitate all the calculations.
One of the good tools for modeling software applications is the unified modeling
language (UML). See, for example, the works [
Figure 3 contains a use case diagram that describes the main functionality of the
software application proposed for design. “Use case is a high-stage description of what
the approach is meant to do, whose purpose is to capture the approach
requirements” [
Figure 3 shows that the software application proposed for design should assess the quality of online education in a particular higher education institution, and (when necessary) conduct a similar comparative analysis for several educational institutions. 8
Conclusions E-learning provides its participants with several undeniable benefits, but it is not free of some essential drawbacks.
With the growing demand for e-learning, the problem of online education quality assessment is becoming increasingly important. This assessment should be carried out based on a scientific methodology using modern tools of economic and mathematical modeling and information technology. The authors believe that in a higher education institution the quality of e-learning as a whole or a particular online education program can be evaluated using the projectbased approach, i. e. regarding an online education program (or a set of programs) as an investment project and assessing the efficiency of the given project for its core participants – the higher education institution and its students. The obtained performance indicators will constitute the objective assessment of the quality of e-learning in a particular higher education institution. These indicators are the response of the socio-economic system to the quality of online education.
Following the proposed approach, it is possible to talk about high-quality e-learning in a higher education institution subject to the conditions as follows: the commercial component for all online education project participants should be non-negative; the online education project will have a positive impact on the process of strategic development of the higher education institution and the student; the online education project will have a positive impact on the reputation of the higher education institution and the reputation of the student (graduate) as a specialist in a particular subject area.
To estimate the values of the weight coefficients that define the importance of the opinions of target groups, the authors propose using a scheme based on Fishburn sequences.
The proposed methodology for assessing the quality of online education in higher schools should form the basis of the corresponding software application since all the calculations within the proposed methodology are rather time-consuming.
We believe that the proposed approach to assessing the quality of online education is reasonable. This approach can add to the existing practices for evaluating online education quality.
The study has been carried out with partial financial support from the Russian Foundation for Basic Research, project No 18-010-00688.