This article presents a theoretical review and empirical study of the assessment of the quality of BA and MA online education during the Russian military invasion of Ukraine in 2022. A qualitative theoretical analysis and comparison are made with a study of higher education quality assessment across Ukraine during the war in 2022 (N = 12019). The article analyzes the tools and structure of student feedback on the evaluation of the educational process. The undergraduate and graduate programs are modified based on the analysis of this data. In the Ukrainian higher education space, the National Qualifications Framework requirements for each specialty have been under development since 2014. The National Qualifications Framework developed a list of interdisciplinary competencies that should be transferred to academic courses. According to the Ukrainian higher education standard, universities must provide empirical evidence that their students actually acquire these competencies. Baseline data on the assessment of the quality of online education by student teachers during the war in Ukraine is presented using the case study of the Kriviy Rih State Pedagogical University (N = 688). Further modifications of the questionnaire to assess the quality of teaching in order to improve data-driven decision-making and testing ethics are proposed.
of higher education in Ukraine. An investigation of students’ attitudes towards higher education in 2022 was commissioned by the Friedrich Naumann Foundation for Freedom. Information on 26 parameters was gathered; for example, level of education by gender, state of residence (internal migration, migration abroad, residence), the format of university education (online synchronous/asynchronous, hybrid, in classrooms, suspended), and the content of education. According to the study “Higher Education in the Eyes of Students” (N = 12019), 94.75% of students are aimed at obtaining higher education, and only 5.25% of university students afirmatively answered that education is meaningless [4]. This indicates the priority of the quality of education for students.
It should be noted that according to all-Ukrainian data, teaching at universities resumed mainly in online format (73.34%). In particular, students answered that the university education was resumed in online format synchronously (55.8%), resumed in online format asynchronously (18.06%), resumed in mixed online format (synchronously and asynchronously) (23.25%) and other options – 3.41% (resumed, part of the classes are conducted online, and part – in person in the classrooms, the educational process is suspended) [4].
In response to the need for Ukrainian universities to realize the online format of learning in the condition of the invasion of the Russian army into the territory of Ukraine and the annexation of Ukrainian territories in February 2022, all universities had to shift to Multi-access learning [5] and HyFlex learning [6]. However, it remains unclear how efective teaching and learning have been in this ”trial mode” of synchronous online courses compared to face-to-face or even asynchronous online formats [6].
According to Beatty [7], “the format is HyFlex (hybrid + flexible), which explains the key principles of HyFlex courses as (a) allowing students to choose how to attend classes, (b) ofering equivalent learning activities in all forms, (c) using the same learning objects for all students, (d) providing students with technology and skills to participate in all forms, and (e) using authentic assessments. According to its proponents, the HyFlex model is more student-centered and more lfexible than standard mixed-mode classrooms because students can choose how to adapt their learning needs to the course environment”.
This creates a request for crisis management and decision-making based on the analysis of the quality of online teaching in wartime.
As evidenced by the answers to the question “I think more often than before about whether the right specialty I have chosen” – only 25.03% have doubts about the chosen profession. At the same time, only 7.2% answered that “At the first opportunity I will change my specialty”. External motivation in studying is observed in 10.82% of students (“If it were not for the insistence of my parents, I would have already quit studying”).
Everything indicates that it is especially important for students to receive a quality education as a condition for their stable future, on the one hand. On the other hand, for teachers, there is a need for flexible transformation of teaching to ensure a high level of education. To assess the quality of online teaching in wartime, it is especially important to develop sensitive tools for feedback. At the same time, these evaluation tools should be convenient for processing large amounts of feedback data and optimizing the learning process.
This creates the first research question: Is multiple types of testing methods (valid, reliable, and fair) for assessing the quality of teaching in order to improve data-driven decision-making and ethics in a crisis situation?
The Center for Innovative Development of Higher Education of the National Academy of Educational Sciences of Ukraine and the Center for Social and Humanitarian Research of V. N. Karazin Kharkiv National University implemented a large-scale project “Students of Higher Education Institutions about their studies” during two months of the 1st semester of 2021 using an anonymous online survey, in which more than 2.7 thousand bachelor’s and master’s students of 39 higher education institutions in all regions of Ukraine took part.
The parameters of the sample meet the requirements of representativeness, which allows extrapolation of the findings to the general population of students of higher education institutions in Ukraine. Almost two-thirds (62%) are satisfied with the level of education quality, and accordingly, one-third (33%) are dissatisfied with only certain aspects of education. In particular, dissatisfaction is caused by teaching methods using Internet communications and computer technologies [8].
At the same time, the majority of students note that in the educational process, teachers apply the latest methods – electronic presentations, use Internet content, online services, video conferencing, etc. To organize training, university teachers use various means of digital communication. At the same time, 30% of students point to the obsolescence of the material, and the lack of new scientific information in the curricula, including 25% of students being dissatisfied with the attitude of teachers towards students, and the level of available communication. The survey shows that the most common online classes are conducted using the Zoom platform, less often – Google Meet, and communication between teachers and students often takes place in groups using the two most popular applications – Viber and Telegram.
Moodle [9] and Google Classroom [10] learning management systems are used to organize the distance educational process, manage it and ensure interaction between teachers and students. Saukh et al. [8] points to the urgent need to improve the digital competence of teachers, which should not be reduced to setting tasks and checking their implementation. It is argued that universities should develop a specific program of psychological support for participants in the educational process and the transition to various forms of distance learning.
The second research question is: What are the ways of anti-crisis digital transformation of teaching at the university in the conditions of military operations based on decision-making based on survey data?
Today, data-driven decision-making in higher education is a component of goal-setting in university management regarding curricula, choice of subjects, and types of teaching to improve student performance, course recommendation, analysis of learning patterns, prediction of dropout, enhancing the eficiency of teachers, and reducing administrative burden [ 11]. There are several uncomfortable issues for data-driven decision-making in higher education (figure 1).
Data collection is possible with the use of a number of educational systems and platforms, which are reviewed in the article “Big Educational Data & Analytics: Overview, Architecture and Challenges” by Ang et al. [13], who note that: “Educational data is rapidly being collected and generated in the higher education ecosystem, which encompasses various systems and platforms such as course management and learning management systems (LMS), massive open online courses (MOOCs), OpenCourseWare (OCW), open educational resources (OERs), and social media sites such as Twitter, Facebook, YouTube, and personal learning environments (PLEs). Learning Management Systems (LMS) are education management platforms for administering, delivering, tracking and reporting on curricula and courses. Moodle is one of the most popular open source LMS options available today. Other examples of LMS are Canvas, Sakai, ATutor, Eliademy. Forma LMS, Dokeos and OpenOLAT. The concept of LMS originated from e-learning”.
According to a review by Ang et al. [13], one of the challenges every institution faces is how to improve student retention and reduce dropout (figure 2). The authors point to two emerging areas for improving student retention and reducing dropouts through data analysis: dropout prediction; and the development of academic early warning systems.
Data-driven decision-making in Higher education is a relevant and rapidly developing area in Ukraine. In particular, in accordance with the policy of digitalization of the state in 20222023, the platform “Digital Transformation of Higher, Professional Pre-Higher and Professional (Vocational) Education (e-University)” is being developed on an ongoing basis in Ukraine. The aim of the project is to automate the admission campaign, organize the recruitment and training (internship) of foreigners and stateless persons, order educational documents and Europeanstyle supplements to them, introduce electronic licensing, modernize the Unified State Electronic Database on Education, create and modernize a unified electronic system for monitoring the employment of graduates [14].
In turn, the service includes electronic licensing in the field of education (e-licensing), which is based on the work of a separate body for evaluating the work of the educational institution – the National agency for higher education quality assurance. According to the Strategy of the National agency for higher education quality assurance to 2022, one of the goals is to harmonize standards and develop criteria for ensuring the quality of higher education based on best international and national practices [15].
According to the Strategy for the Development of Higher Education [15], it provides for the protection of higher education from academic dishonesty, the provision of low-quality educational services, lack of interaction and trust between stakeholders (figure 3).
To solve this problem, it is advisable to use Data-driven decision-making and educational data mining (EDM) – interrelated areas in educational research. The National agency for higher education quality assurance provides recommendations on the views on the current state of testing the quality of higher education and typical questionnaires for online surveys. However, there is currently no comprehensive study or literature review on models of questionnaires and grading scales for the Ukrainian higher education system.
One of the tasks of the reform of higher education in Ukraine transforms the structure and mechanisms of student feedback on the educational process. Bachelor’s and Master’s degree programs are created on the basis of the analysis of these data, which gives the right to employment. In the space of Ukrainian higher education, there has been the development of requirements for the National Qualifications Framework of each specialty since 2014. For the National Qualifications Framework, a list of interdisciplinary competencies must be transferred into academic courses. According to the Ukrainian Higher Education Standard, universities must provide empirical evidence that their students are actually acquiring these competencies [16].
If we analyze the mechanism proposed by the National agency for higher education quality assurance (methodology of independent, external, on-site evaluation of the quality of legal education in Ukraine), the questionnaire was implemented from the American educational system and contains an assessment of the university’s policy on the quality of evaluation of educational, scientific and educational-professional programs and should include a generalized assessment of the uniqueness of the program using qualitative analysis (focus groups) proposed by the administration, faculty, and employers and quantitative (statistical) quality assessment.
This survey consists of two parts. In the first section, respondents rated each statement on
a 5-point Likert scale from “(
At this stage of educational practices in Ukraine, educational programs of Ukrainian universities do not yet have such unified standardized assessment tools and evidence base for analysis in the form of scientific publications, on the one hand. On the other hand, there is a nationwide mechanism of mandatory evaluation, in accordance with the standard procedure for licensing educational programs for university students, conducted by the National agency for higher education quality assurance every 4-5 years for each specialty. Thus, in the Ukrainian educational space it is relevant to create a scientifically based research standardized algorithm for monitoring the quality of teaching disciplines and the type of motivation for choosing disciplines by students and postgraduates.
Analyzing the tools to improve student retention and reduce dropout through data analysis and learning analytics presented in figure 2, it should be emphasized that the Model Questionnaire (testing) of the Strategy of the National agency for higher education quality assurance to 2022 (presented in figure 3) refers to the analysis of academic problems, financial problems and psychosocial problems. However, it requires theoretical understanding and practical implementation of testing the assessment of the motivational component in data collection.
The work “Konstruktion eines Fragebogens zur Erfassung der Lehrkompetenz (LeKo)” by Thiel et al. [17] presents the technique of constructing and validating questionnaires to determine the level of teaching competence. It was developed at the Freie Universität Berlin as a universal tool for teaching evaluation without any connection to the subject. The article examines diferent approaches to teaching evaluation. It is emphasized that there are several approaches to this process, each of which represents a certain perspective of vision: from the position of experts, from the position of students, and from the position of the teacher himself (self-evaluation).
Evaluating teaching competence from any perspective has both advantages and disadvantages compared to the other two, so the quality of the evaluation depends primarily on the choice of an efective validation tool, one of which, according to the authors, is the educational success of the students. In addition, for an adequate evaluation, the researchers emphasize, both areas of competence (Kompetenzbereiche) and quality criteria and indicators (Qualitätsdimensionen) must be clearly defined [ 17]. Based on the analysis of scientific research of previous years, the generalization of already developed forms of questionnaires for evaluating the quality of education, the authors of the article proposed the form of the questionnaire developed by them for determining the level of teaching competence, theoretically substantiated and experimentally tested its efectiveness [17].
In the structure of the methodology “Das Berliner Evaluationsinstrument für selbsteingeschätzte, studentische Kompetenzen (BEvaKomp)” conducted by Braun et al. [18], contains a questionnaire of 29 items consisting of six subscales, measuring, success in knowledge acquisition, methodology/presentation, communication/collaboration competence and personal competence. The results show suficient reliability, and factor analysis proves that the instrument establishes diferences between competencies and has suficient discriminant validity. Two ideas went into the methodology for creating the instrument: 1. Why are measures of subjective competence important in assessing the educational success of a course of study? 2. How can subjective competency acquisition be recorded as a result of attending a course? According to Braun et al. [18], in order to assess the contribution of individual courses to subjective competency acquisition visibly, students are asked to rate the increase in competency as a result of attending the course.
A distinction is made between professional competence, methodological competence, social competence, and personal competence. When applied to university education, the four areas of competence mean the following: 1. Professional competence means that students expand their knowledge, understanding, application, and analytical skills [19]. 2. Methodological competence refers to an individual’s ability to efectively plan work and master appropriate work methods [20]. 3. Social competence refers to the ability to realize one’s own goals while protecting the interests of others [21]. Some authors distinguish it in communication and cooperation skills [22]. 4. Personal competence describes an individual’s productive attitude toward learning and self-development.
The authors focused on these four domains of competence in developing an instrument to measure subjective skill acquisition while attending university courses. A detailed definition of the diferent competency domains can be found in [18].
The main conclusions of the validation of the method are that, according to the Bologna Process, course evaluations should be based on competencies and not – as has been almost exclusively the case so far – on student satisfaction with the teaching staf. The tool developed difers significantly from other assessment tools.
This adaptive approach is an extension of Staufenbiel and Hartz [23], who developed diferent scales for course types (lectures, seminars, exercises). Thus, a descriptive assessment of courselevel filter questions also allows conclusions to be drawn about the extent to which courses of an interactive nature (seminars, exercises) enable the development of social skills. For example, the evaluation criterion for seminars can be the number of students’ oral presentations (descriptive evaluation of the filter question on communication skills) [24].
The development of BEvaKomp is understood as the first step toward outcomes-based and competency-based course assessment, as required by university reforms under the Bologna Process. BEvaKomp records self-assessment in non-specialized competency areas. Future studies will need to investigate the extent to which subjective competencies can be validated using “more rigorous criteria” such as academic performance and professional success after graduation [24].
In the study by Braun et al. [18], Braun and Leidner [24], a systematic analysis of content and construct validity was carried out to create the methodology “Assessment in Higher Education: Self-Assessment of Competencies (HEsaCom)”. The methodological diference of the methodology from similar ones is that students evaluate diferent aspects of their growth and benefits from the course, not how much they liked the teacher.
Braun and Leidner [24] acknowledge that when using the self-report method, it is impossible to use strict standardization criteria, as well as the fact that students’ self-assessment of teaching quality can be distorted by students’ liking or disliking of the teacher, the grades they received in the course. The authors rightly believe that leveling this kind of distortion is possible if students self-assess the growth of their own cross-cutting competencies. When developing HEsaCom, the authors conducted an analysis of construct validity, in particular, diferentiating between the construct of self-assessment of competence acquisition and the construct of teaching and teacher assessment.
The general concept of the methodology was the hypothesis that it is able to selectively measure the acquisition of competencies according to students’ own assessments as opposed to the assessment of teaching quality. Confirmatory factor analysis (CFA) was conducted to hypothetically distinguish between the constructs of competence acquisition and course satisfaction. The obtained factor models made it possible to substantiate how the HEsaCom methodology difers from a similar questionnaire by Westermann et al. [25], which is focused on the teacher’s personality and assessment of his/her professional behavior. Among the scales of the HEsaCom methodology are “Acquisition of knowledge”, “Systematic competence”, “Presentation competence”, “Communication competence”, “Competence in cooperation” and “Personal competence” [18].
The ideas of Dochy and Alexander [26], Bloom [27], Wentzel [28], Braun and Clarke [29], Erpenbeck [30] concepts were used for the meaningful validation of individual scales. The factor analysis conducted by the researchers indicates that the structure of the association of the six competency areas with academic course satisfaction is confirmed by the global and local CFA fit indices [ 31, 32], and thus it is established that the acquisition of cross-cutting competencies by students and their satisfaction with teaching are diferent constructs. Thus, an important conclusion of the design and testing of the methodology is that the highest priority of the criterion of quality of higher education such as the acquisition of competencies is approved. Therefore, the assessment of pedagogical behavior and teacher’s personality is a less important criterion. The main theoretical contribution and novelty of the HEsaCom methodology is that it allows for the evaluation of the course focused on the result and competence. The use of such a criterion allows, according to Braun et al. [18], to encourage teachers to think more about the results of students than about their own pedagogical behavior and it provides an opportunity to introduce a culture of result orientation.
In particular, Dübbers and Schmidt-Dafy [33] proposes a thematic survey intervention to encourage students’ self-motivation. The study indicates that course instructors and experts in DBDM (Motivation: Non-regulation, External regulation, Introjected regulation, Identified regulation, Integrated regulation, Intrinsic regulation). Thus, exam-related reasons were formed into items measuring external regulation, as exams are often a strong and visible external influence on university students. Items representing more independent forms of motivation were formed (e.g., “I studied because this content will be useful for me as a future teacher”), and items representing less independent forms of motivation (e.g., “I prepared for the final knowledge test to get confirmation of active participation in the lecture”).
The scores were integrated into an overall score, the so-called Relative Autonomy Index: RAI = (internal regulation + integrated regulation + identified regulation) – (introjected regulation + external regulation motivation). This concise and well-validated measure allows comparing diferences in the degree of self-determination of students’ motivation with a single score, with higher scores indicating more self-determined motivation in relation to less self-determined motivation [33].
Also, ethics regarding the content of the survey should be observed. For example, in the study by Piskurska [34] on the types of monitoring of the quality of education, the author aptly notes that, first of all, most questionnaires assess the quality of a teacher’s work and focus on the personal qualities and behavior of the teacher. The author proposes an algorithm for monitoring the quality of education in the form of 6 stages of research. Also, the researcher rightly notes that based on the data of student surveys to demand changes in the personality of the teacher is a violation of human rights, but it is quite legitimate for the administration of a higher education institution to change the work of the teacher (facilitation skills, studentcentered learning, methods of stimulating individual work). Given this circumstance, the objects of monitoring in the form of questions can only be those requirements for the organization of the educational process, which are fixed in the regulatory documents of each particular higher education institution: regulations on the organization of the educational process, job descriptions of teachers, regulations on the current control of student performance and interim certification, etc. Also, Piskurska [34] emphasizes that the content of monitoring should include only those requirements, the implementation of which is likely to be problematic.
In another study on the significance and complexity of test tasks during the evaluation of the performance of university teachers, an analysis of existing approaches to the construction of rating systems, educational standards and curricula are proposed to generalize theoretical questions of the essence, structure, functions, features of the application of test control of student learning success during the evaluation of professional activities of university teachers. The publication confirmed the possibility of evaluating the teacher’s pedagogical activity based on the results of monitoring the success of student’s studies and developed a method for calculating the importance of a test task without an expert [35].
Machova et al. [36] and others in their study proposed a cloud architecture for analyzing educational data from the Moodle system in the cloud using Apache Hadoop. This study also presents data mining in an e-learning environment from an educational perspective.
In the pilot (N = 62) we used a questionnaire for online surveys proposed by the National
agency for higher education quality assurance (Methodology of independent, external, on-site
evaluation of the quality of legal education in Ukraine). The questionnaire was implemented
from the American educational system. This survey consists of two parts. In the first section,
respondents rated each statement on a 5-point Likert scale from “(
However, the proposed questionnaire on assessing the quality of teaching subjects in military and distance learning environments proved to be unreliable when piloted in April 2022 because students 1) did not complete it to completion (32%), 2) ignored the test (16%), and 3) answered the same way on a 5-point scale with a score of 5 (27%), thereby creating a positive feedback gala (as indicated by the high test completion rate: 4-6 minutes) without realizing the questions asked.
Therefore, it was decided to modify the questionnaire to highlight the most important
indicators for the university consisting of 20 questions on a 5-point scale, 3 questions with
answer options, and 1 open-ended question. For all psychological variables, respondents gave
answers on a 5-point Likert scale from “(
Issues that pointed to the successful Design of Programs: 1.1. I need discipline for my future professional activity. 1.2. The discipline contains useful material. 1.3. The discipline is logically connected with other disciplines. 1.4. The discipline contributes to the formation of skills and abilities. 1.20. In general, it was interesting for me to master this discipline.
In this study, every student was asked questions that indicated the style of teaching (Teaching and Assessment): 1.6. The professor is fluent in educational material and modern scientific information; 1.7. The professor motivates students to independently search for information in depth; 1.8. The professor clearly formulates the goals and the training plan; 1.9. The educational material is presented in an accessible and interesting way; 1.10. The professor uses the latest interactive teaching methods.
The possible benefits from Student-centered learning were evaluated with the following questions: 1.12. I have the desire to continue studying with this teacher (other disciplines, coursework, qualification work); 1.13. The teacher is open and friendly with students; 1.14. I always had the opportunity to turn to the teacher for clarification or advice (items: 1.12-14) and The teacher is tactful and knows how to establish contact with students (item: 1.16).
The questions that indicated the Learning Resources and Student Support, Certification: 1.5. The discipline is provided with the necessary textbooks and teaching materials. 1.11. The teacher clearly defines the criteria for assessing students’ knowledge. 1.17. The professor always conducts classes on time and according to the schedule. 1.18. Distance learning was well organized by the professor. 1.19. But, in my opinion, it would be more correct to teach the discipline personally.
The Cronbach’s alpha of this scale (Teaching and Assessment) was 0.69, minimum 1, maximum 5, a range of 1.12, variance of 0.20, and a mean of 4.37. Test–retest reliability with an average interval of 2 weeks showed a Pearson’s correlation of r = 0.71 for the Design of Programs, r = 0.79 for the Teaching and Assessment, r = 0.67 for Student-centered learning, and r = 0.72 – Learning Resources and Student Support, Certification.
Perceived benefits related to studying goals were then examined using 4 items that were asked to assess the dificulty of the academic discipline (question 2). Is discipline contributed to a specific studying goal such as a clearer professional self-image, or increased professional hard skills? 3. Estimate the ratio of the amount of educational material and the time allocated for its processing. 4. When did the teacher provide you with information about the procedure and evaluation criteria?
The internal consistency of these items was 0.82 measured with Cronbach’s alpha. The test-retest reliability with an average interval of 2 weeks for each item/each treatment goal varied from 0.51 to 0.70 measured with Spearman’s rho ( ). These items were analyzed on a single-item level because we wanted to explore each goal by itself and therefore were not considered as a scale.
We plan to realize our design of research on the quality of teaching in Universities in war
conditions. The respondents’ perceptions of the status of the quality of teaching are analyzed
based on four indicators: (
The survey of the quality of teaching in Universities in war conditions (computer-based
surveys powered by Google forms) contained 20 questions and 4 statements about perceived
benefits from courses, associated factors, and participants of studying process behavioral
characteristics. These answers were scored on a 5-point Likert scale from “(
Data analysis will be analyzed using SPSS 20. Generally, type of analyses is conducted:
descriptive statistics (means and standard deviations) were used to identify the level of explained
variables (
We assessed 122 academic courses in 11 teacher preparation with on areas (kindergarten teachers, elementary school teachers, and high school test–retest An electronic questionnaire form (appendix A) was emailed to the dean’s ofices of 7 departments and distributed to all students in groups at the end of the summer semester, but before final exams.
Data collection for the current project was held from June to July 2022 using the Computer Assisted Web Interviewing method. The research concentrated on attitudes towards studying among students who continue their studies at university (BA, MA programs) or were students of Kryvyi Rih State University as of February 24, 2022, regardless of their status at the time of the survey (N = 688). In total, 688 students who participated in the research answered our questionnaire at the end of the year (N = 688; 81.5% female, 28.5% male). The sample included students without stratification by specialties and courses.
Data were analyzed with IBM SPSS 20. Firstly, we investigated the level of the computer web interview method.
The majority of students (87%) rate the disciplines that were taught from February to June 2022 as necessary for their future professional activities (figure 4). This coincides with the all-Ukrainian study of assessing the attitude of students toward higher education [4]. 1.1-1.4, 1.20 questions (Design and Approval of Programs) keep the same proportions and trends: 1.1. I need discipline for my future professional activity ( = 4.5, = 0.95). 1.2. The discipline contains useful material ( = 4.6105, = 0.77779). 1.3. The discipline is logically connected with other disciplines ( = 4.4390, = 0.93876). 1.4. Discipline contributes to the formation of skills and abilities ( = 4.5581, = 0.85284). 1.20. In general, it was interesting for me to master this discipline ( = 4.4433, = 1.01645).
The questions that indicated the style of teaching (Teaching and Assessment): 1.6. The professor is fluent in educational material and modern scientific information ( = 4.7195, = 0.66833); 1.7. The professor motivates students to independently search for information in depth ( = 4.4971,
= 0.92514); 1.8. The professor clearly formulates the goals and the training plan ( = 4.5901, = 0.82924); 1.9. The educational material is presented in an accessible and interesting way ( = 4.4637, = 1.01595); 1.10. The professor uses the latest interactive teaching methods ( = 4.2762, = 1.10149).
The possible benefits from Student-centered learning, teaching, and assessment: 1.11. The teacher clearly defines the criteria for assessing students’ knowledge ( = disciplines, term papers, qualification papers) ( = 4.5872, = 0.91308); 1.13. The professor is open and friendly with students ( = 4.5131, = 0.98155); 1.14. I have always had the opportunity to turn to the professor for clarification or advice; 1.15. The teacher always fairly, objectively, and transparently assesses the knowledge of students ( = 4.5596, = 0.90088); 1.16. The professor is tactful and knows how to establish contact with students ( = 4.5596, = 0.89804).
The questions that indicated the Learning Resources and Student Support, Certification: 1.5. The discipline is provided with the necessary textbooks and teaching materials ( = 4.2747, = 1.06628); 1.17. The professor always conducts classes on time and according to the schedule ( = 4.5916, = .88183); 1.18. Distance learning was well organized by the professor ( = 4.5916, = 0.88183). Only statement 1.19. But, in our opinion, it would be more correct to teach the discipline personally ( = 2.7267, = 1.42989) has a declining trend since the study was conducted during hostilities (table 1).
An important condition for improving the method of self-assessment and feedback to students regarding the efectiveness of learning a course is establishing the criterion validity of such a method [18]. A criterion that ensures reliable and valid feedback results is considered to be self-assessment by students in the mastery of end-to-end competencies provided by the educational program of the specialty. Such self-evaluation shifts the focus of the criterion of the efectiveness of the educational program and individual disciplines to the student’s general vision of his progress in general and special competencies, in particular those that demonstrate the general orientation of the program to the problems of the educational process and subjects of education, the significance of the results of independent work on the Moodle platform in the individual learning trajectory of students.
Is multiple types of testing methods (valid, reliable, and fair) for assessing the quality of teaching in order to improve data-driven decision-making and ethics in a crisis situation?
Just like the Report “Higher Education through the Eyes of Students”, where students rate higher education very highly. In our study, the averages for all questions ranged from 3.72-4.724 average.
The majority of students (figure 4) reported that they had quite a lot of benefits from subjects in general with = 4.23. Only 2.2% of students reported that they perceived no benefits in general. For benefits relating to Teaching and Assessment, 65.2% of the students reported these benefits = 4.31. The results of the scale Learning Resources and Student Support, Certification repeated measures showed that highly (79.3%, = 4.38). This was also the case for Student-centered learning, teaching, and assessment (55.4%, = 4.28).
Considering the results of the study should analyze the content of the survey. In the conditions of hostilities, recommending changes in the behavior of a teacher is a violation of personal boundaries. Therefore, the subsequent objects of monitoring can only be those requirements for the organization of the educational process that are enshrined in the regulatory documentation of each particular university: regulations on the organization of the educational process, job descriptions, teachers, regulations on the current monitoring of student progress and intermediate certification, etc.
It is advisable to include in the content of the questionnaire only those requirements, the fulfillment of which is likely to be problematic. The questionnaire should also contain general information about the language of teaching the discipline, the percentage of attendance at classes in the academic discipline, and the number of points based on the results of the current progress control.
When compiling a questionnaire to assess the qualiity of teaching, it is worth considering the model of Slavin [37], who singles out four important criteria: quality of instructions, relevance, motivation and optimization of time with relevant indicators. In addition, although scientists point to the non-ideal nature of the model, the ambiguity of assigning indicators to one or another criterion for determining the quality of education, we believe that the author of the model defines the core of each criterion quite precisely: quality of Instruction (structure, clarity, comprehensibility, variable forms of teaching, speed, use of media, exercise intensity, scope of material, performance expectations and level of dificulty), appropriateness (dificulties degree, adaptivity, problem sensitivity, individual support and counseling, individualization, support orientation), incentives (teaching content and learning goals, avoiding performance anxiety, arousing interest and curiosity, afirmation and reinforcement, positive social climate), time (time available, used time, content orientation, classroom management).
Another well-known model consists of three dimensions, namely: eficiency of class management, potential of the lesson for cognitive activation and constructive support of the learners [38].
The analysis of the theoretical basis for determining the quality of education, the technology of conducting surveys to assess the quality of education in all its aspects allows us to draw certain conclusions about the structure and content of the questionnaire, which should include questions about the quality of teaching and the organization of the educational process: the quality of instruction, the presentation of the material, its presentation; use of media resources; quality of educational content, its scientific level; time manager; ability to interest, motivate to study the subject.
What are the ways of anti-crisis digital transformation of teaching at the university in the conditions of hostilities based on decision-making based on survey data?
The second modification of the questionnaire gave a 96 test completion rate with a completion time of 12-23 minutes. However, the results of the survey do not give obvious statistical results, which in the conditions of crisis management provide a basis for forming changes in curricula, schedules, number of hours and planning of training retraining for faculty. This encourages further development of the types of testing and data collection based on stratification of the sample according to the disciplines that received the highest marks in groups of students and disciplines that received the lowest marks in groups. This implies a diferent research design.
For further research based on the data, we prospectively assume that with the help of the survey it is likely to: 1) to form a rating of the discipline: relevance in terms of hard skills formation, the novelty of the material, accessibility of educational material, preference for hybrid, synchronous / asynchronous learning, transparency of course evaluation, complexity of the course. 2) to form a component of the teacher rating: student-centered teaching, teaching and support styles. 3) to identify empirically the indicators of student dropout and loss of motivation in the training system. 4) to form promising areas of digital transformation of teaching that support the motivational component of student participation.
The datasets generated for this study are available on request to the corresponding authors.
Ethical review and approval were not required for the study on human participants in accordance with the local legislation and institutional requirements. The participants provided their written informed consent to participate in this study.
Instruction “Dear Student / Dear Student! The University Administration invites you to take part in a survey on the level and quality of teaching disciplines. Your answers will help to improve the educational process and improve the quality of education at the university. The survey is conducted anonymously. Thank you in advance for participating in the survey!”
20 questions on a 5-point scale, 3 questions with answer options, and 1 open-ended question.
For all psychological variables, respondents gave answers on a 5-point Likert scale from “(
1.2. The discipline contains useful material. 1.3. The discipline is logically connected with other disciplines. 1.4. The discipline contributes to the formation of skills and abilities. 1.20. In general, it was interesting for me to master this discipline.
Style of teaching (Teaching and Assessment):
Learning Resources and Student Support, Certification: