=Paper=
{{Paper
|id=Vol-2770/paper17
|storemode=property
|title=The Influence Online Learning Quality Criteria Selection on Negentropy
|pdfUrl=https://ceur-ws.org/Vol-2770/paper17.pdf
|volume=Vol-2770
|authors=Olga Andryushkova,Sergey Grigoriev
}}
==The Influence Online Learning Quality Criteria Selection on Negentropy==
https://ceur-ws.org/Vol-2770/paper17.pdf
The Influence Online Learning Quality Criteria Selection
on Negentropy
Olga Andryushkova1[0000-0002-1566-3427] and Sergey Grigoriev2[0000-0002-0034-9224]
1
Lomonosov Moscow State University, Faculty of Chemistry, 1, GSP-1, 1-3 Leninskiye Gory,
119991 Moscow, Russia
o.andryushkova@gmail.com
2
Moscow City University, 2nd Selskohozoyastvenny proezd, 4/1, 129226
Moscow, Russia
grigorsg@mgpu.ru
Abstract. Methods for assessing the quality of education are discussed, includ-
ing the possibility of predicting learning outcomes based on the calculation of
negentropy. Negentropy is used as an integral informational index, demonstrat-
ing objective assessment of the learning model used. We propose to use the
emergent learning model as a generalized projection of the learning process
with a reasonable fusion of e-learning and traditional learning. To create a mul-
ti-criteria system for assessing the quality of education we propose to evaluate
all components of the pedagogical system in interaction with each other, at the
first and all levels of the hierarchical levels. We also propose to define as a spe-
cial category of programs in the system that interacts with other elements of the
system, and thus the system is organized as “student-centered”. The proposed
methodology for predicting and assessing academic performance is based on
building on building a hierarchical structure of multi-criteria learning quality
system. At the first stage, we selected the main ones that affect the quality of
education, which are organized in the first level of the Ishikawa diagram. Then,
based on the knowledge of experts, we assigned each criterion an appropriate
coefficient of importance for the quality of training. At the second stage, the
same was done for the second stage criterion of the third level of importance. In
cases where it was not possible to unambiguously estimate the coefficient of
importance, we present the system of equations for calculating the membership
function of a fuzzy set. At the third stage, the integral values of negentropy
were calculated for three university blended courses and for a model situation.
Keywords: e-learning, Emergent Learning, Fuzzy Set, Quality of Learning,
Negentropy.
1 Introduction
The boom in online learning caused by the COVID-19 pandemic is unprecedented
in the history of distance learning. A massive shift to distance learning technologies
Copyright © 2020 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0
International (CC BY 4.0).
Proceedings of the 4th International Conference on Informatization of Education and E-learning Methodology:
Digital Technologies in Education (IEELM-DTE 2020), Krasnoyarsk, Russia, October 6-9, 2020.
(DLT) at different levels of education was impossible to imagine recently, especially
in schools.
On the one hand, the shift gives a wide prospect for conducting an experiment in
literally “field” conditions and analyzing the learning outcomes at a scale, which was
previously impossible to dream of. On the other hand, one cannot but take into ac-
count the circumstances when the distance learning technology was literally “thrown
at the embrasure” in order to keep the educational process afloat. In the context of the
pandemic, when there was no time to comprehensively think over the structure of
courses’ content and learning scenarios, choice of tools, training of teachers and stu-
dents, were there an opportunity to get a quality learning process? Let's honestly ad-
mit that no, it was not possible – as well as production and implementation of any
complex product in conditions of a shortage of time, necessary funds and resources.
But distance learning technologies have already been designated guilty of improper
performance of the educational process. We will discuss some of the most common
mistakes that have been committed in massive shift to implementation of distance
learning technologies and measures for providing high-quality emergent learning in
the future.
2 Features of “сoronalearning”
Probably, the first thing that was done almost everywhere was an attempt to com-
pletely transfer the face-to-face learning without any changes to online mode through
video activities.
Traditional face-to-face classes timing was applied to all learning formats: lessons,
seminars and lectures. Moreover, it was required to follow all traditional lessons tim-
ings in the webinar mode: if a traditional lesson lasts 45 minutes, then a webinar
should also last 45 minutes. In some educational organizations, timing compliances
was strictly monitored by administration. It was not taken into account that such re-
quirements were violated the State sanitary and epidemiological rules and regulations,
especially for primary schools.
We will formulate here only a few questions, answering which, perhaps, it will be
more clear how to build the educational process using distance learning technologies
and e-learning in modern conditions:
─ Is strict adherence to face-to face classes timing in transition to distance learning a
prerequisite for the high-quality learning?
─ Do lectures in obligatory synchronous online mode guarantee the quality of educa-
tion?
─ Who should form the scenario of the educational process: a teacher, an administra-
tor of the e-learning platform used an administrator of the educational organiza-
tion?
─ Is it legitimate to present the same requirements for the learning process and use
the same type of distance learning models for all levels of education and fields of
study – natural sciences, humanities, economics, engineering, medicine, etc.?
The history of distance learning technologies and e-learning counts more than one
decade, the fundamental principles of DLT and EL developed include the following:
carefully planned design of the learning process, taking into account the target group,
form and level of education; developed online course (preferably tested), which meets
the requirements of DLT and EL; obligatory guide/guidebook for the course| both for
students and teachers; registration of students’ independent work in the e-learning
platform used; convenience of work with online courses for both teachers and stu-
dents. For example, in case of large number of students located in different time
zones, listening or watching video materials occurs at a time when it is convenient for
each student. Insisting on the simultaneous participation of tens and hundreds of stu-
dents, for example, in online lecture does not seem entirely reasonable. Moreover,
within the MOOC framework it has already been convincingly shown how video
materials can be used in the most effective ways; availability of technological, organi-
zational and consulting support (administrators, curators, tutors, coaches, mentors) for
teachers and students; participation of trained teachers (often authors and designers of
their own e-course) with high level of ICT and distance learning competence, which
include the following: methods of online education; educational software; e-learning
technologies; relevant ICT and e-learning terminology; compliance of curriculum
requirements, teaching materials, and learning conditions; integration e-learning re-
sources into educational process; methods of developing and managing online-courses
in electronic learning systems; communication in electronic learning environment;
writing scripts for 3D, VRML-models, etc.; group work communication tools; meth-
ods of students motivation for efficient learning in electronic learning environment.
Serious problems with teachers’ ICT, DLT, and e-learning competencies were
highlighted precisely in the shift to distance learning technologies in the context of the
pandemic, aggravated by time pressure. These problems seems to be unexpected in
the situation of implementation in recent years of numerous projects in the field of
digitalization of education, development of state requirements to e-learning environ-
ments and resources and variety of programs for teachers’ ICT and e-learning train-
ing. But in a result teachers should have fulfilled the requirements of administration:
to work online according to face-to-face classes timetable and answer students' ques-
tions by e-mail. Do these requirements apply to current trends in e-learning, blended
learning or distance learning? Not at all. However, distance technologies are blamed
that the educational process does not line up, videolessons do not give the expected
learning outcome and turn the teacher's work into an “online hard labor”.
And, finally, there is one more problem – insufficient number of e-courses, devel-
oped by the teachers. These courses could be used in distance learning (not only in
pandemic context), in blended learning as digital component of full-time traditional
courses, as the extra students’ feedback channel and resource for practicing e-teaching
skills.
3 Approaches to assessing quality of online learning
In modern realities, the issues of assessing the quality of the educational process
using online technologies remain relevant and are actively discussed in professional
communities, including from the point of view of assessing the effectiveness of using
information and communication technologies in the educational process. Based on
publications [1-6], we can conclude that the most discussed issues are the selection,
grouping and ranking of criteria for the quality of education.
The term “quality of the educational process” is proposed to assess the compliance
of the educational process with a certain quality standard that is used in the educa-
tional organization and takes into account all the components of the pedagogical sys-
tem. From this point of view, the assessment of the quality of combined or emergent
learning [7] is undoubtedly determined by the quality of all components of the educa-
tional process: teachers, students, electronic educational resources, electronic learning
environments, technical and technological support of learning and availability and
provision of laboratory workshops. As an additional criterion is proposed to take into
account the external requirements for the educational program, set in the Federal State
Educational Standard or in an independently established educational standard.
The requirements for the structure of resource support at a university described in
detail are schematically presented in Figure 1.
Fig. 1. The structure of the resource support of educational process in conditions of EL and
DLT
If we analyze each of the five compulsory components of comprehensive support
for a successful educational process, then the reasons for obtaining a “surrogate” ra-
ther than a distance or blended learning process become obvious.
Briefly, we can formulate some typical obstacles to implementation of DLT and
EL: technological and psychological unpreparedness of some teachers and students to
online learning; lack of proven methods and experience in organizing a large-scale
educational process online; insufficient amount of quality e-learning courses; attempts
to use the maximum of the Internet technological capabilities without reasonable jus-
tification.
Some modified forms of organization of the educational process in the conditions
of EL and DLT are presented in Table 1. As shown in Table 1, there is no need to use
exclusively video lessons in a synchronous mode for conducting classes, there is a
wide range of alternative modified forms. Moreover, it should be emphasized that
considerable experience has already been accumulated in their practical use, and
therefore, it is possible to avoid typical mistakes made at the initial stages of devel-
opment and implementation of various learning models.
Table 1. Correspondence of traditional and modified forms of organization of the learning
process (LP)
Traditional Types of forms of organization of Modified forms of organ-
forms of LP the LP ization of the LP
organization
Lecture Depending on:
─ MOOC format based on
─ didactic goals and place in the video: chronicle; studio,
LP: introductory, course setting,
current, final, overview ─ stream lecture,
─ way of carrying out: information- ─ lecture in VR,
al (classic), problematic, binary ─ webinar mode (synchro-
discussions, provocative lectures, nous or asynchronous -
lectures-conferences, lectures- recording)
consultations,
─ video lecture,
─ goals: learning, informational,
educating, developing, ─ web lecture
─ content: academic, popular sci- ─ internal or external navi-
ence gation through Internet
resources
Seminar Depending on the method of con-
ducting: ─ webinar (synchronous),
─ seminar-conversation, ─ online seminar in the
Moodle module with off-
─ seminar-discussion, line peer review;
─ seminar-conference, ─ webinar in on- or off-line
modes,
─ problem seminar,
─ wiki seminar;
─ seminar press conference,
─ work with animation
─ brainstorming workshop,
models and simulators,
─ specialization seminar,
─ work with the glossary,
─ peer-learning workshop ─ work in VR or with AR
Laboratory In a specially equipped room
work (with devices, construction kits, ─ a virtual laboratory (in
machine tools, tools, reagents, uten- VR or with AR) for work-
sils, etc.) for: ing with simulators for
real installations, research
─ mastering the technique of exper- objects, experimental
iment, conditions;
─ experimental confirmation of ─ remote laboratory (hard-
theoretical statements; ware and software com-
plex),
─ formation of the ability to solve
practical problems by setting up ─ electronic laboratory
experiments, complex,
─ formation and development of ─ interactive manuals
skills to work with devices,
equipment, installations;
─ formation of the ability to safely
work with chemical
─ reagents and labware
─ visual presentation of the condi-
tions for performing the experi-
ment, measuring instruments
necessary for a real experiment;
─ selection of the optimal parame-
ters for the experiment;
─ obtaining skills in drawing up
plans, schemes for organizing a
laboratory experiment
Individual Doing homework, solving tasks.
independent Preparation for seminars, labora- ─ working with an online
work tory and practical work, tests, course (text, photo, video
Olympiads and conferences materials),
─ completing training tasks,
─ work with simulators and
training modules,
─ work with test systems in
self-test mode
Formative Exam or credit:
and summa- oral by tickets, ─ remote testing with proc-
tive assess- written by tickets, toring,
ment Tests, ─ video survey, webinar,
execution and defense of a crea-
tive task, ─ completing tasks on ticket
defense of the final qualifying issues within the deadline
work and posting answers to
the site
An integral assessment of quality of the educational process can be performed on
the basis of the identified major categories in the Ishikawa diagram [8, 9], which af-
fect the quality of student learning in the pedagogical system.
Depending on the teaching model used, the form of education, target audience and
indicators of achievement of competencies, the categories of all levels will differ, but
the following regularities should be preserved:
─ the category of the first level should be no more than six or seven, taking into ac-
count the method of forming cause-and-effect relationships when constructing the
Ishikawa diagram;
─ major categories should reflect the main elements of the system under considera-
tion, in this case the pedagogical system;
─ the hierarchy of the system under consideration is a sign of its stability as a dynam-
ic system, and then, the more detailed all the nested categories are revealed, the
more clearly the strengths and weaknesses of the pedagogical system are revealed;
─ a set of categories of all levels provides the emergent properties of the pedagogical
system.
On example of developing a methodology for assessing the quality of the educa-
tional process in Chemistry courses the major categories that affect the quality of the
competencies formed in the course were identified. It was proposed to distinguish
seven such categories [10]. In Figure 2 the major categories that affect the quality of
student learning in the context of online learning technologies are given as an exam-
ple.
Fig. 2. Ishikawa diagram: the first level categories and their weights
The weight coefficients are determined by the normalization method based on ex-
pert assessment.
The maximum values were received for such a category as “Laboratory” (see
Fig. 2).
Then, in the hierarchy of importance, the category “Teacher” follows as a subject
of the pedagogical system, which is beyond doubt since the teacher acts as a designer
and organizer of the educational process and the second-level categories for all other
basic criteria depend on the level of teacher’s competence and motivation.
Each of the major categories can be represented as a set of categories of the second
and third levels. In this work the major category “Student” (including the categories
of the second and third levels that affect student’s level of learning in the context of
an emergent approach to learning) is considered in detail. Table 2 shows the catego-
ries associated with both the student's own psychological and intellectual abilities and
the criteria determined by the external environment: the student group, the education-
al organization and the educational information and learning environment.
Table 2. Criteria of the second and third level for the category “Student”
Attractiveness of the specialty or correct vocational
guidance.
Adaptation in the learning environment and commu-
nication in the group
Motivation to
Student The degree of matching of expectation from the
learn
learning process and a real situation
The use of gamification elements (indication of
learning progress, etc.)
Demand for the results of each student's activity in
the success of the group / team
Convenience of schedule
Basic training formed at the previous stage of study
and confirmed at additional entrance tests or tasks.
Ability to Willingness to change cognitive structures by receiv-
learn (formation ing and processing information
of specified The level of development of cognitive processes:
competencies) perception, thinking, memory, attention, speech
The level of development of the emotional-volitional
sphere: perseverance, purposefulness, poise.
Electronic component: device and gadgets, software,
Internet access
Availability
Electronic learning environment (system, platform)
of resources
Full-time component: equipped laboratories, materi-
als, reagents, instruments, dishes
Teacher, tutor, coach, curator, site adminis-
trator
Communica- Among students within the group and withing the
tion and support similar groups
Teamwork on joint projects, peer review and evalua-
tion of works
Psychologi- Information and communication competences
cal readiness to User friendliness of the learning environment inter-
work in elec- face
tronic learning Matching the psychological characteristics of the
environments personality with the type of activity.
(ELE))
One of the ways to assess the quality of online teaching is an anonymous survey of
students.
The results of survey of Geology students for the General Chemistry course are
shown in Figure 3. The assessment was carried out on a five-point scale and demon-
strated good accessibility of the course; high involvement of students in online learn-
ing based on the electronic educational and methodological complex (EEMK); satis-
factory indicators for the electronic learning system (ELE) used, taking into account
that the platform was unfamiliar to students.
Fig. 3. ELMC and ELE quality indicators for General chemistry course
A comparative analysis of qualitative indicators for assessment of academic per-
formance for five academic years of work with the course and implementation of
point-ranking assessment system is shown in Figure 4. As data in Figure 4 show,
there is a decrease in the percentage of students who did not take the exam with prac-
tically insignificant fluctuations in the quality of academic performance and average
score. The results obtained, in our opinion, can be explained by the fact that the com-
bined use of the online course and point-ranking assessment system leads to an addi-
tive effect. This effect is expressed, on the one hand, for students – in the need to
systematically work independently with the course materials and to pass tests to check
the preparation for laboratory work; on the other hand, for the teacher – in require-
ments to update the learning outcomes data using the electronic journal module and
the course materials on regular basis as well as actively communicate with students
using feedback modules.
Fig. 4. Geology students’ learning outcomes for the General Chemistry course
4 Calculation of criteria values based on fuzzy sets
The issues of using and calculating weighting factors for various groups of catego-
ries remain open, although the processing of an array of expert data on taking into
account the significance of different-level criteria in a hierarchical pedagogical sys-
tem can be considered as a classic problem of using fuzzy set algorithms. It is shown
[10-16] that fuzzy set algorithms are used to solve various applied problems, includ-
ing the design of information systems for automatic control of knowledge and student
progress, for automatic information extraction from texts and in other areas where it is
necessary to formally describe the concepts or phenomena that have ambiguous or
imprecise characteristics. From this point of view, the base of expert opinions on the
importance of influencing the quality of student learning of such categories of the first
level as teacher; student; educational and methodological support; technical and tech-
nological support; methodological and technological support; external requirements
for the educational program and the equipment of the laboratory and practical base –
is a database for processing using fuzzy set algorithms.
The calculation of the numerical value for the criteria of the second level of the
Ishikawa diagram was carried out on the basis of systems of equations of fuzzy sets.
Based on the criteria of the second and third levels for all basic categories, assessed
dichotomously or using fuzzy set algorithms, when the membership function has an
asymmetric two-sided Gaussian distribution [13], it became possible to compare the
quality of the educational process for Chemistry courses for different majors.
Negentropy (J) was calculated according to the equation:
J=wi·ki,
where ki is the numerical value of the criterion, and wi is its weight coefficient. For
the calculation of negentropy, the methodology which made it possible to obtain
comparative data on the influence of “corona learning” on the educational process
carried out with DLT and EE [17] was used.
Figure 5 shows the results of the calculated value of negentropy, taking into ac-
count all the major criteria that describe educational processes for two majors – Geol-
ogy and Chemical Sciences and for a model situation when all the criteria are the
most favorable and assessed with the maximum score.
Fig. 5. The calculated value of negentropy for the educational process in the Chemistry
courses
Based on the set of criteria selected for the analysis, it is possible to make predic-
tions about the achievement of a given level of quality of the educational process for
various majors.
The desired assessment can be carried out by choosing those criteria that are de-
termined by the internal quality standard of the university / faculty / institute. To ob-
serve the phenomenon of emergentism [18] in the applied learning model, built on a
combination of the basic components of traditional classical education and e-learning
elements, in addition to specifying the set of major criteria that directly affect the
quality of student learning it is necessary to define a detailed hierarchy of criteria of
the second and third levels, taking into account all the nuances of the educational
process. In this case, it is possible to characterize the educational process by calculat-
ing negentropy – an integral indicator characterizing all elements of the pedagogical
system.
References
1. Andreev, A. A. Russian open educational resources and mass open remote courses. Higher
Education in Russia 6, 150–155 (2014).
2. Andreev, A. A. The quality of online learning. Proc. 4th Int. Conf. “E-learning in continu-
ous education”. Ulyanovsk, UlSTU, pp. 340–344. (2017).
3. Andreev, A. A. Assessment of quality of online courses. Territory of Science] 1, 20–26.
(2015).
4. Bovtenko M.A., Parshukova G.B. Subject MOOCs as component of language learning en-
vironment Advances in Intelligent Systems and Computing 677, 122-127 (2018).
5. Kruglov, V. I., Gorlenko, O. A., Mozhayeva, T. P. Formation and development of quality
systems of educational institutions. Higher Education in Russia 12, 46–51 (2015).
6. Kaldybaev, S. K., Beyshenaliev, A. B. Quality of educational process in structure of quali-
ty of education. Advances in Current Natural Sciences 7, 90–97 (2015).
7. Andryushkova, O. V., Grigoriev, S. G. Calculation of the negentropy and weight coeffi-
cients of multicriteria estimates on the basis of fuzzy sets. Informatics and Education
1(300), 40-49 (2019).
8. Ishikawa K. Guide to Quality Control. Tokyo, Asian Productivity Organization (1976).
9. Isikava Diagramma Cause-and-Effect-Diagram, http://www.up-
pro.ru/encyclopedia/diagramma-isikavy.html 2020/11/05.
10. Tsvetkov, V. Ya. Information Uncertainty and Information Certainty in Information Sci-
ence. Information technology 4, 3–7 (2015).
11. Lotockii, V.L. Entropiya i negentropiya. Prospects of science and education 1(25), 20-23
(2017).
12. Tsvetkov, V. Ya. The entropy analysis of data in physics, biology and the equipment.
SPbGETU, Saint-Petersburg (2015).
13. Zadeh, L. A. Fuzzy sets. Information and Control. 8(3), 338–353 (1965).
14. Ryzhov, P. A. Elements of the theory of fuzzy sets and fuzziness measurements. Moscow:
Dialog-MGU (1998).
15. Leonenkov, A. V. Fuzzy simulation in MATLAB and fuzzyTECH. BKHV-Peterburg,
Saint-Petersburg (2005).
16. Vostroknutov, I., Kaneda, Y. The possibilities of using modern Casio CG-50 graphing cal-
culators for volumetric and complex calculations. 13th International CONFERENCE on
Theory and Application of Fuzzy Systems and Soft Computing, vol. 896, pp. 702–708,
(2018).
17. Andryushkova, O. V., Grigoriev, S. G. Emergent learning in the information and education
environment: monograph. Informatika i obrazovanie, Moscow (2018).
18. Yulina N.S. Consciousness, rediction, causality. Voprosu filosofii 12, 127-143 (2010).