The publication is devoted to the analysis of the e-learning system in crisis situations. The necessity of calculating the dependence of the satisfaction level of using the system on the possibility of communicating with the tutor and receiving positive evaluations on the final study results is substantiated. Correlation coefficients are in the range from 0.88 to 0.99, which indicates a close relationship between these results. Obtaining quantitative data is obtained through the method of surveying students at the beginning and end of each module. The structure of the adaptive e-learning system and the principle of knowledge assessment of learners are described. The need to achieve the level of digital literacy is substantiated. The research was conducted using an electronic learning system international project "dComFra"/"Framework of digital competences for Ukrainian teachers and other citizens" of the Erasmus + program under the direction KA2: Capacity Building in the Field of Higher Education Higher Education.
learner blended learning, e-learning, information technology, dComFra project, emotional state of the
The development of mankind has always been accompanied by booms and busts, new achievements, cultural revolutions, epidemics and wars. Certainly, human always remained a main actor of such events. Thousands of knowledgeable, smart and wise individuals normally drive the humanity progress. Past centuries are rich with inventions and events, sourced by scientists and political figures. And with each step of progress, human life becomes faster and requires increasing amount of knowledge.
Today’s reality without gadgets, information technologies and complex mechanisms controlled at a distance even hard to imagine. Smart homes and technological processes controlled by IoT systems have entered into everyday life and became a part of our wellbeing.
Last century, as well as current decade human is a progressive individual who is influenced by external factors and forced to make multidirectional and variative decisions.1].
Nevertheless, each crisis, each epidemic or war was ending by peace, further development and technological progress. It was facilitated by a person's ability to learn under guidance and self-study. New knowledge acquiring and obtaining new skills are progress engine. Children acquire the knowledge in family, kindergarten, school (full-time education). Young people go to professional, secondary and higher education institutions (face-to-face learning and blended learning which includes the distance learning and interactive tools usage, especially in higher education institutions). On employment stage, learning continues, but certainly modify. A mixed education model in form of
2022 Copyright for this paper by its authors. supervised courses in combination with Internet-trainings is used more. A more or less clear separation by types of training was in effect until 2019. The Covid-19 pandemic made its corrections.
More than 1.5 billion students have been affected by the pandemic, with youth and the most vulnerable students the hardest hit. Variety of the gains achieved by the way to 2030 Education Agenda goals have been lost [2]. The pandemic opened a new stage in the development of education technologies and learning tools. One of these main tools has become e-learning within support of the various communication platforms for discussions and tutting. From 2019 to 2022, a huge number of research have been conducted and scientific works have been published on the impact of the Covid-19 Pandemic. Some of them were analised, which are confirm the necessity of e-learning using, its advantages and shortcoming. The total number of publications can be divided into several types: elearning problems, learning tools and the psychological state of learners and teachers (Figure 1). Based on the work 4], the experience of distance learning (conducted during the first five months of quarantine) caused a corresponding change of opinion about the nature of the education experience, as well as the desire to continue such experience of distance and blended learning in slightly more than 50% of respondents. The work 5] also confirms the positive people's attitude who has shifted to elearning. From the researched sample point of view, the level of attitude towards test programs in the virtual classroom is high. There is a positive, strong and statistically significant correlation between the attitude towards the use of tests in the virtual classroom and the motivations for learning 5]. For example, e-learning is recognized as a useful education approach and it becomes more and more popular among the undergraduate clinical medicine 6].
The respondents reported that 96.1% of lectures, 85.5% of seminars, and 40.0% of clinical classes were implemented by e-learning. The lectures conducted by e-learning were assessed as good and very good by 78.4% and seminars by 51.2% of respondents. While the clinical classes conducted by elearning were assessed as bad and very bad by 62.9% of respondents. The most frequently indicated limitations of e-learning were the quality of the content and available materials (26.9%), restrictions in direct contact with the lecturer (19.6%), Internet connection (16.8%), and living environment (13.8%). Only 4% of the students had to buy or update computer equipment. Any other limitations were indicated by 9.7% of the respondents. Survey was performed nationwide via the Internet from 30th November 2020 to 10th February 2021. Six hundred fifteen (615) medical students completed the survey 7].
Research results 8] conducted at a university in Great Britain show that transactional distance affects participation, and e-learning capital affects the engagement of learning skills. Our findings suggest that if universities are to continue online learning for students who previously studied on campus, they should provide access to the infrastructure and training to use the online ecosystem so as not to place students at a disadvantage. Further investment in student capital for e-learning, such as "Help", FAQ and adaptation of existing resources, is also needed to support this key influence on the engagement of study skills. Computer self-efficiance is a factor related to engagement in digital learning technologies. Investment in students' e-learning capital is needed as a key influence on learning skills engagement and should be reflected in university policy. Educators should focus on showing the value of participation, and academic advisors should review practical tips on study skills for online learners. This cohort effect of agreements between students should be further encouraged in online learning, including facilitating the exchange of learning approaches 8]. In works 9-11] the results of studies of the transformation from learning at desks to digital learning are given. The theory about students' perception of this form is confirmed, but there are certain psychological problems in both students and teachers. Crisis situations are a significant factor in the sudden transition to e-learning.
E-learning, which creates problems such as isolation, demotivation and learning burnout, has become the main education tool, due to the COVID-19 pandemic. Challenges with e-learning can be counterbalanced by peer-to-peer (P2P) learning. There is a renewed interest in user adoption of elearning through different platforms, as the sudden and long-term transition to e-learning shows different results compared to what was known before the pandemic 12].
One of the problems of the transformation to e-learning is the training of educators who has to teach disciplines in educational institutions, to work with interactive materials, interfaces of communication platforms, and to fill up virtual classrooms with education materials. The works 13-14] highlight the problems of encouraging students to e-learning and the problems of assessing the practical skills of students. Research has shown that teachers' perception of student engagement affect how they interact with students, as well as student grades. It is critical to understand how teachers perceive engagement, not least in an online environment, to inform about practice and research 14].
It is desirable to assess the problems and crisis states of learners using the e-learning satisfaction scale 15, 18], which has several gradations. In 15] they propose a developed and tested e-learning satisfaction scale (eLSS). The eLSS consists of 17 items with three factors: content, interface, and communication. Convergent validity with eLSS was supported, and Cronbach's alpha was 0.93 [15].
Another aspect of the transformation to e-learning is the tutting of educators for the role of virtual coach. Works 16, 17, 19] describe the issue of teachers training as the main component of the e-learning process. Digital technologies are used to offer the individual education experience. In this case children will engage in self-motivated, personally meaningful exploration at home using mobile apps for literacy. In the same time trainers will scaffold children's learning asynchronously based on their playing data 19]. The main reasons to create the digital learning environment are Ukrainian educators digital competences training challenge, in accordance to DigComp framework and necessity to equip non-IT students, as well as secondary school pupils with digital skills. Sure, to install, fine-tune, structure learning approach and fill up the digital learning environment within the content the efforts of various professionals are needed. Such creating team should include: methodologists, system administrator, digital learning environment administrator, software developers and experts in each digital competence for content development.
Since the early 2019, the rapid development of digital learning environments and simple e-learning platforms began mostly everywhere in the world 20-23]. Ukraine was chosen as the research region. Today it's a country affected by both the COVID-19 Pandemic and Russian invasion. It requires the educational sector to force strength to introduce different level electronic learning systems. The problem of electronic learning systems using in crisis situations related to the impossibility of visiting educational institutions is urgent. The degree of satisfaction with such system will be assessed based on the results of a survey of users of various categories. First, it is necessary to conduct an analysis of the digital literacy of the population based on representative samples.
Let's consider this problem in detail using the example of Ukraine. In 2019, 41,400,881 people lived on the territory of Ukraine as of January 1, 2020, of which 1,869,743 were aged 15-19, and 2,094,913 were aged 20-24 3]. As of January 1, 2022, the population decreased by 34,500,000 people [24]. Detailed information is presented in Table 1. The decrease in the population of Ukraine from January 2020 to 2022 by an average of 20% is caused by crisis phenomena in the state. According to the statistics of state services 24], on average, about 15% of the population changed their place of residence. These are citizens of Ukraine of various ages, among whom there are people who are studying and need elearning tools.
In accordance with Table 1, all age groups of population can be involved in the educational process in form of e-learning, except of children aged 0-4 years. For children aged 5 to 14, the system requires limited access to learning platforms in accordance with sanitary standards. Persons from 15 to 45 years old are able to use e-learning platforms in a self-convenient mode and to obtain all necessary learning information. People over 45 and under 59 require a permanent supervisor during the work with the elearning platform. People over 60 need to work in groups under the tutor’s supervision. These conclusions were made based on the survey results, which includes representatives’ groups of different age category. For the survey, a sample of 100 people was taken in a percentage ratio of 5 age categories. A diagram of the survey results is shown on Figure 2.
One of the simplest conclusion: dependence of the constant tutor presence or absence during elearning process shows us most digitally literate age group - 15 to 45 years.
The study of the use analysis of the electronic learning system was carried out using the digital learning environment located at dole.fit.knu.ua. This digital learning environment was developed based on the results of the international project "dComFra"/"Framework of digital competences for Ukrainian teachers and other citizens" of the Erasmus + program under the direction KA2: Capacity Building in the Field of Higher Education. Digital learning environment’s content structure is shown on the Figure 3. Analysing today’s approach to teaching digital competences for non-IT students in Ukraine, has shown its narrowness and inadequacy to meet needs of students with different knowledge level. Taking in consideration EU and UA Digital Competence Frameworks, international standard for digital literacy – ICDL and results of Erasmus plus CBHE project “dComFra – digital Competence framework for Ukrainian teachers and other citizens”, No 598236-EPP-1-2018-1-LT-EPPKA2-CBHE-SP the authors propose a new paradigm of teaching digital competences for non-IT students. The main advantage of the new approach is the higher effectiveness of teaching digital competences for non-IT students with different knowledge levels in one training group 27].
The system should include web and mobile interfaces, appropriately scoped server subsystems and up-to-date dynamic learning content. Functionality of the system should also provide modern concepts of learning and interaction in the following areas: student-student, student-teacher, teacher-teacher. The implementation of the described concept will contribute to the development of digital competences of different target groups: teachers, unemployed persons, persons in need of retraining and so on. Each of these target groups will have its own, customized digital competence system. The feature of this system is its dynamism, since it changes in content and even content type over time as well as its refinement according to the needs of the audience. The implementation of the concept will build a comprehensive roadmap to support the development of digital competences in learning for many years to come. This will enhance the role of interactive communications and digital learning in the community, attract more and more target groups to ICT, and deploy mobile applications in digital competence training 26].
The results of the survey on the use of e-learning platforms 100 y t i t an 50 u Q 0 0-4 05-14 15-45 46-59
Si f S, p,T , t number of persons working with the e-learning platform, % the number of persons requiring constant tutor participation,%
Each module contains theoretical materials, presentations, practical works, intermediate tests on the topic of the module. The module ends with a final test and a satisfaction questionnaire. Formal limited time period is set for the any final test.
The degree of correctness of each answer is set in one of two types: 1) correct or incorrect (1 or 0); 2) the quantity of points obtained by choosing this option can take an integer or a fractional number.
The difficulty of the task is evaluated in two variants: determining the quantitative grade in the form of a certain number of points or percentages, or using fuzzy logic.
Testing developed by the "question-answer" method. As default, test tasks are provided one-by-one in sequential order, testee doesn’t have possibility to jump between questions, but this option can be changed in the system. After each task answer, its correctness is evaluated. Based on the results of the test, knowledge is evaluated by the 100-point scale, which can also be considered as fuzzy. The level of preparation S depends on the current level of preparation Si, the percentage of correct answers p, the difficulty of the task T, and the time to answer the task – t. All the above parameters are fuzzy.
We will assume that: the higher percentage of correct answers, the higher level of knowledge; the higher complexity of the task, the higher level of skills; the longer time to answer question, the lower competence’s level.
The control of questions answering time is provided to eliminate the possibility of not answering the questions independently (searching for the correct answer in the textbook, synopsis, etc.) – the grade will still be low, even with correct answers. On the other hand, if a person is not use hints, but simply pondered for a long time, this indicates an insufficient knowledge level and the grade should be reduced. To define an appropriate testing time tmax several factors should be taken in consideration: testee should have enough time to read the question and answer options, think about and choose the answer.
The response time can be taken into account as follows:
1 (tresp tmax ) , if the answer is correct Ri 100 0 , if the answer is not correct when Ri – result of the answer to the ith task.
If Ri > 1, then Ri = 1.
If Ri 0 , then Ri 0 ,
when tresp – task response time;
tmax – the time during which the decrease in assessment does not occur.
(
The parameter tmax is set constant for all test tasks or is calculated depending on the complexity of the tasks, i.e. t miax f (Ti ) . It makes sense to spend more time on a difficult task than on a less difficult one. The time of testing – tmax depends on the individual capabilities of the person being tested 28].
In the process of using the electronic learning system, communication between the student and the teacher occurs according to the following diagram (Figure 4).
The system takes over part of the teacher's functions and creates positive motivation among students, since computer testing is more psychologically comfortable compared to traditional survey forms.
The proposed structure of the adaptive electronic learning system allows: to increase the effectiveness of learning due to openness, multifunctionality and adaptability; to evaluate the selfacquired knowledge in study process due to reviewing and clarifying questions, as well as works, which is certainly reflected in the total number of received points.
The advantages of such a system include: 1) the ability to edit the test bank; 2) reducing teacher’s workload due to some functions transfer to the adaptive subsystem of knowledge control: a) instant test results without any additional efforts from the teacher’s side; b) objectivity of assessment; c) confidentiality during anonymous testing.
The learning process model scheme allows to meet an individual approach in the learning process to each learning object using adaptive methods (Figure 5); to implement an individual approach to the learning object in current topic study process.
In case of the test result higher or equal to minimum allowed points number, the system informs user for the grade, and in the same time offers to learn next block of theoretical and practical materials, which in its turn will be accomplished by the next test.
The module considers as completed and appropriate digital competence gained after the final test successful passing. The final test includes some number of the questions from each passed topic, chosen randomly, but from all difficulty level's sections. For the feedback, further improvement of the system and its content, understanding degree of satisfaction with the process of adaptive learning, the trainee passes module's satisfaction questionnaire.
The group of 130 people working in crisis conditions was chosen as representative. The crisis conditions were due to the real martial law: periodic air raid alerts and communications failure, permanent danger sensation and nervousness. All trainee got access to the platform and to 4 initial level modules of the adaptive electronic learning system (digital learning environment) on September 15, 2022. Before to proceed with learning process, the group passed a questionnaire containing 10 questions. The questionnaire determines the level of learning expectations. Table 2 shows summary data of the stochastic analysis by response type. The satisfaction degree is ranked from 1 to 5, which corresponds to the values: "dissatisfied", "indifferent", "rather satisfied", "satisfied", "very satisfied".
After the Module 1 studying and succesful passing of the final test, the module's satisfaction survey was conducted. Table 3 shows summary data of the stochastic analysis by response type for this survey. dissatisfied indifferent rather satisfied satisfied very satisfied dissatisfied indifferent rather satisfied satisfied very satisfied
Changes in the satisfaction indicators of module 1 trainees % n i se 40% s n po 30% s re l s ta20% ee t
o ian teh10% r t f fo o 0% r e b m u n e h T 2% 0% 1 15% 3% 2 35% 24% 3
36% 27%
37% 21% 4 5
Dependence of the learning satisfaction level on
receiving a positive evaluation
The dependence of the level of satisfaction on the received assessment is close, because the correlation coefficient of these values is 0.997307031 units. During the month, there was an increase in critical indicators: the increase in the incidence of covid 2019, the destruction of civilian and power infrastructure, cold spell and reduction of the daylight, increasing dependence on electricity (mostly unavailable and coming by unstable schedule) and Internet connection (more offten disappearing with electric power). Table 5 shows the dependences for the survey on module 2 and Table 6 – on module 3, taking into consideration critical conditions' continual tightening. The dependence of the values has decreased. The correlation coefficient is 0,884920357. evaluation based on the results of the final test. Module 3.
dissatisfied indifferent rather satisfied satisfied very satisfied
Total
3
45
60
18
4
130
2
34
49
29
16
130
(
To check the reliability of the correlation relationship, we calculate the measure of inaccuracy of the correlation coefficient according to the formula: where: – satisfaction levels number (in our case 5); – correlation value (calculated separately for each module); ( − 2) – degree of freedom according to the Student’s t-test.
The reliability of the correlation coefficient between the pair of characteristics "receiving a positive final module’s assessment" - "degree of learning satisfaction" is calculated using the formula:
The calculation results are presented in the table form (Table 7).
The values of the measure of inaccuracy of the correlation coefficient and the reliability coefficient of
0,997 0,8849 0,92
Measure of inaccuracy of the correlation coefficient
0,42 0,2688 0,221 The calculated reliability of the correlation coefficient t 2,373 3,29 4,17
> 0,1 < 0,01 < 0,01
To calculate the probability, which confirms the reliability of the correlation coefficient between the pair of characteristics "receiving a positive final module’s assessment" - "degree of learning satisfaction", let’s compare the obtained calculated value with the t value of the standard correlation coefficient according to the Student's table. The degree of freedom is 3. Standard values are 2.35 for the significance level р = 0,1; 3,18 for р = 0,05; 5,84 for р = 0,01 and 12,9 for р = 0,001.
Thus, for Module 1, where
> 0,1 the reliability of the correlation coefficient between the pair of characteristics "receiving a positive final module’s assessment" - "degree of learning satisfaction" is confirmed. For Modules 2 and 3 the reliability of the correlation coefficient between the pair of characteristics "receiving a positive final module’s assessment" - "degree of learning satisfaction" occurs with a probability of
The explanation for the adaptive electronic learning system use satisfaction level continual decrease is in the human body response to a crisis situation and the ability of a person to adapt to the such situation until peak insensitivity occurs. Nevertheless, in such critical conditions, no any other education techniques would be better and most of them are not applicable in general.
Dependencies of the level of satisfaction in
changing crisis situations S E SN60 O SEP50 R S EE40 N I RA30 T F O20 R E BM10 U N E 0 H T
The graphs confirm the impact of the crisis situation on e-learning. 48 respondents had the highest level of satisfaction during completing the first module (in more or less stable situation), just 11 respondents saved the same highest level of satisfaction during completing the second module (under conditions of increased crisis state), but the number of respondents with the highest level of satisfaction had increased during completing the third module (under conditions of being in an intensified crisis state for a certain period). One of the posiible conclusions is the attitude of the respondents to the learning process depends on the learning conditions, including external factors, and possibilities to communicate with the tutor.
The task of digital literacy for Ukrainian citizens has high priority. To solve this task, the digital learning system was developed due to the dComFra project. There are a lot of publications describing learning / digital learning / obtaining digital skills in standard conditions. Since 2019 the world has entered into the crisis conditions and certainly education sector also. It’s reflected in the publications reviewed by the authors in this article. The war in Ukraine is strengthen crisis situation exponentially and rapidly changing people's perception of the world around them. The questionnaires of people, who were trained and have successfully passed final testing, were used for the research. Each trainee noted the desire for the maximum grade. Therefore, the pair of characteristics "receiving a positive final module’s assessment" - "degree of learning satisfaction" was chosen for the research. These characteristics strength of relationship was evaluated by means of mathematical statistics, namely: correlation. The article describes the principle of assessment formation and the system and means of communication between the tutor and the trainee. The summarized results of the survey of the trainees regarding their satisfaction with the course are shown in Tables 2-6. A graph of the dependence of the course satisfaction level and positive assessment receiving in crisis situations of various degrees is plotted (Module 1 – the lowest degree of crisis situation, Module 2 – the highest degree of crisis situation, Module 3 – an adapted level after the highest degree of crisis situation). According to the graph in Figure 8: 95 (47+48) trainees of 130 (74%) received the maximum "excellent" points. The satisfaction level has an increase in Module 1 and on a contrary sharp decline in Module 2. Going through the Module 2 under severe critical conditions gives the lowest level of learning satisfaction (less than 10 people), in spite of 48 people received 90+ points.
The strength of relationship of the characteristics pair "receiving a positive final module’s assessment" - "degree of learning satisfaction" varies from 0.88 to 0.99 in accordance with the calculations according to tables 4-6. The relationship is strong, despite the varying strength of the critical learning conditions. The reliability of the correlation is confirmed by probability calculations according to the Student's test with a degree of freedom of 8 for 5 satisfaction levels.
Underline – the learning system satisfaction level depends on the learning conditions: from standard to extremely critical.
In context of Erasmus plus CBHE project “dComFra – digital Competence framework for Ukrainian teachers and other citizens”, № 598236-EPP-1-2018-1-LT-EPPKA2-CBHE-SP: The European Commission's support for the production of this publication does not constitute an endorsement of the contents, which reflect the views only of the authors, and the Commission cannot be held responsible for any use which may be made of the information contained therein.