=Paper=
{{Paper
|id=Vol-2740/20200262
|storemode=property
|title=Improvement of Teaching Quality in the View of a Resource-Based
Approach
|pdfUrl=https://ceur-ws.org/Vol-2740/20200262.pdf
|volume=Vol-2740
|authors=Nataliia Komleva,Vira Liubchenko,Svitlana Zinovatna
|dblpUrl=https://dblp.org/rec/conf/icteri/KomlevaLZ20
}}
==Improvement of Teaching Quality in the View of a Resource-Based
Approach==
https://ceur-ws.org/Vol-2740/20200262.pdf
Improvement of Teaching Quality in the View of a
Resource-Based Approach
Nataliia Komleva1[0000-0001-9627-8530], Vira Liubchenko1[0000-0002-4611-7832] and Svitlana Zi-
novatna1[0000-0002-9190-6486]
1Odesa National Polytechnic University, 1 Shevchenko av., Odesa, 65044, Ukraine
komleva@opu.ua, lvv@opu.ua, zinovatnaya.svetlana@opu.ua
Abstract. In the paper, a way to improve the teaching quality by applying a re-
source-based approach is discussed. The solution includes a model of teaching
quality, a method of student-centered teaching, and means for analyzing and
managing the unobservable quality characteristics. We propose to consider the
education process as a system with negative feedback, where the deviation of the
values of the educational process characteristics from the normal ones diagnoses
the type of the problem that interferes with the normal state of the educational
process. The formalization of the unobservable quality characteristics of the ed-
ucational process is carried out. The paper considers a means for measuring the
values of these characteristics in different scales and provides examples of meas-
uring. We also provide a formalized description of the teaching process and a set
of questions for a student to evaluate teaching. An algorithm for the creation of a
questionnaire, which covers the characteristics of various components of the
teaching process, has been developed. Finally, we describe the task of informa-
tional diagnostics for the results of a student’s evaluation and the catalog of teach-
ing tactics as a means for systematic formation of quality-improving activities.
Keywords: Resource-Based Approach, Student Evaluation of Teaching, Qual-
ity Characteristics, Questionnaire Completeness, Teaching Tactics.
1 Introduction
The adoption of the Law on Higher Education in 2014 launched the reform of Higher
Education in Ukraine. The reform created enormous challenges for universities and
teaching staff. The academic and administrative staff of higher education institutions
had to change the mindset concerned with different business processes.
In the paper, we focus our attention on the improvement of teaching quality. Differ-
ent aspects of education, such as forms and methods of teaching, updating educational
content, types of control procedures, and evaluation criteria, affect the integral quality
of the study program. It is a well-known rule taken into account in “Regulations on
Accreditation of Study Programs in Higher Education” developed by the Ukrainian Na-
tional Agency for Higher Education Quality Assurance and approved by an order of the
Ministry of Education and Science of Ukraine.
Copyright © 2020 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
Teaching quality means different things for different people. For example, students
expect high-quality content provided in a clear, friendly, and creative manner. Profes-
sors contribute to achieving the objectives and learning outcomes. Experts of method-
ological departments look for appropriate study materials, etc. In the paper, we restrict
the set of stakeholders by the principal ones – students and professors.
For many years, students’ evaluation of teaching (SET) has been the most common
process employed at universities to evaluate faculty-teaching quality. However, many
problems – such as mental biases, participation rates, validity, reliability, and several
other related issues remain actual. For example, in [1], the authors described data pat-
terns: “in SET with lower participation lecturers with positive evaluation receive better
scores and lecturers with bad evaluations are rated worse.” In [2], the authors wrote:
“we find that women systematically receive lower teaching evaluations than their male
colleagues.”
In the paper, we do not focus our attention on the problems of SET. We focus atten-
tion on the activities that enable us to discover the ways for improving teaching perfor-
mance with reliable results, which becomes possible with the use of particular re-
sources. As a resource, we understand any unit of practical knowledge required by a
teacher or academic manager in order to perform a particular task of quality improve-
ment. Resources include everything required for solving a challenge: models, methods,
and means [3]. Models and methods relate to an information part of resources and
means belong to a technological one. Models are our representations of a particular
entity of the educational process. Methods describe the transformations, which are car-
ried out with resources. Means provide the realization of these transformations.
Let us describe the resource-based approach for searching the ways for improving
teaching performance. The remaining part of the paper is organized as follows. Section
2 describes the model of teaching quality with an emphasis on quality measurement
problem. Section 3 presents the method of teaching quality improvement based on
measuring different quality characteristics. Section 4 describes the formal presentation
of the questionnaire structure to obtain full coverage of the teaching quality character-
istics and involves the issue of cataloging the teaching tactics.
2 Related Works
The idea of using feedback from participants to control the teaching quality is not new.
Different kinds of learning processes involve various approaches, tools, and techniques
that should interact harmoniously and effectively. The level of harmony determines the
degree of participants’ satisfaction in the learning process. Anyway, the quality of feed-
back is determined by the quality of the survey. The construction of the survey depends
on the model of the learning process. Since the educational process has a multi-criteria
nature, it is quite difficult for description, which causes the complexity of building ad-
equate models. Let us review some approaches for construction of such models.
The TALIS model assesses the teaching quality basing on the working condition of
teachers and the learning environment [4]. This model takes into account empowering
teaching professionals, including teachers’ satisfaction with their profession and the
current work environment, motivations to become a teacher, and correspondence of a
real salary to the expectations of higher school teachers as to how they should be ap-
preciated.
One of the well-known models – Assessing Quality Teaching Rubrics (AQTR) –
assesses the pre-service teachers’ quality teaching practices [5]. It established a suffi-
ciently high ecological and constructed validity and demonstrated a high degree of con-
sistency. During lessons taught by physical education teachers, it was concluded that
AQTR is a psychometrically supported measure.
However, the assessment of the educational process by teachers and educational in-
stitutions is one side of the coin. The other side is represented by students who can
express their degree of satisfaction with the learning process in which they participated.
The cumulative effects model (CEM) is developed to accomplish this. It assesses the
quality of the teacher’s preparation on the basis of students’ outcomes using survey
tools [6].
TEQAS model is a model in which quality is assessed via assessment of pedagogical
education [7]. Students were surveyed through a questionnaire that covered five quality
variables. The data were analyzed using descriptive and inferential statistics. The re-
search results of pedagogical education in Pakistan have been criticized by the con-
cerned constituencies for excessive quantitative expansion and showed the low quality
of contents, learning environment, processes, and outcomes.
To eliminate the shortcomings of the above models, the Teaching Maturity (TeaM)
Model was created [8]. In this model, the quality of teaching is considered for the edu-
cational process not only in universities but also in primary and secondary schools. The
TeaM model was applied for 19 courses on computer science at Universität Klagenfurt.
The use of the maturity model is possible both by an educational institution to assess
the quality of training and compilation of discipline ratings and by the teacher them-
selves for the purposeful modernization of their course. However, according to the au-
thors themselves, Maturity Model results showed that the extent of correlation between
the Maturity Level of the educational process in a particular discipline and the students’
perception of the course is only weak.
Thus, some of the above models have demonstrated their applicability in practice,
which means that finding ways to describe and model the educational process is an
urgent and promising task. Correction of the shortcomings and limitations of the given
models will allow the proposed resource-based approach to finding ways to increase
the effectiveness of the educational process.
3 The Model of Teaching Quality under Study Program
In [9], the authors claimed that a growing knowledge base had shown that professors
and their instructional approaches are critical factors for the effectiveness and improve-
ment of the educational process. While speaking about teaching quality, we should use
the quality characteristics, which mean inherent characteristics of a professor’s perfor-
mance, that gives some information about an aspect of teaching quality. Let us distin-
guish two groups of characteristics – observed and unobserved in the teaching process.
The correlation between teaching quality and success level of students’ learning is
beyond doubt. The better is the teaching excellence, the higher students’ grades should
be. Therefore, the set of individual grades or integrated indexes calculated on the set
should be considered as observed quality characteristics.
In addition, the effect of the professor’s behavior on students’ achievement is beyond
doubt. The study process reflects the professor’s behavior, which means we should seek
to explain changes in students’ progress by students’ experiences in the classroom or
other study settings. The appropriate aspects of teaching quality are unobserved via
learning. SET is traditionally used to evaluate such quality characteristics, more pre-
cisely, such aspects as:
─ teaching excellence and encouragement, which includes teaching skills, level of con-
tact and involvement, the effectiveness of the course design, assessment and feed-
back in developing students’ knowledge and skills;
─ the learning environment, which includes the effectiveness of study resources in sup-
porting students’ learning and the development of independent study skills;
─ students’ engagement and outcomes, which include self-evaluation of their own
work and progress.
Quality characteristics should be unambiguous. To determine whether a response to
a characteristic is satisfactory, it is necessary to provide the measurement of the char-
acteristic. The purpose of measuring is to reduce subjectivity while monitoring activi-
ties and provide data for analysis.
To achieve the required level of quality characteristic, the professor can use the
teaching tactics. A tactic is a decision of the teaching staff that influences the achieve-
ment of a response to the quality characteristic; tactics directly affect the students’
achievements. The focus of tactics is on a response to a single quality characteristic.
However, tactics can refine other tactics.
Fig. 1 shows the relationship between the entities of the teaching quality model
graphically. Teaching quality characteristics reflect the factors that relate to teaching
quality requirements. As it was mentioned above, there are two types of characteristics:
observed and unobserved via learning. The quality characteristics provide the means
for measuring to determine whether the teaching quality is meeting the required quality
thresholds set by stakeholders. Teaching tactics describe how a given quality charac-
teristic can be achieved.
Fig. 1. The model of teaching quality
For the model, one of the core issues is the measure of unobserved characteristics. SET
instruments typically consist of a set of Likert scale survey questions [10]. Students
select responses on this scale, usually from “strongly agree” (or 5) to “strongly disa-
gree” (or 1), and professors receive a summary report with the mean values for these
responses and possibly the overall mean. The use of the mean value assumes a Gaussian
distribution of responses even though responses may be bi-modal or even tri-modal,
representing differing views of the classroom experience.
However, the Likert scale in itself is categorical, and SET data cannot be evaluated
validly using parametric statistics. These categories differ in quality, not in quantity or
magnitude. In other words, the “interval distance” between the categories is undefined
[11]. For example, any statistical evaluation of categorical data should not include
measures of central tendency like means or averages that are appropriate only for quan-
titative data. An average calculated on categorical data is quite meaningless and mis-
leading; it is not possible to interpret average scores of categories.
The opponents of SET as measures of teaching effectiveness argue that SET has no
or only limited validity as a measure of professors’ teaching effectiveness [12]. Due to
the issues of the scale and influence of irrelevant factors on teaching effectiveness, the
use of SET as a measure of professors’ teaching effectiveness for making high-level
administrative decisions is highly controversial. However, the use of SET as feedback
for professors’ use and making some decisions about teaching quality is not controver-
sial.
4 The Method of Improvement of Teaching Quality in
Educational Process
Let us describe how to encapsulate SET into a student-centered educational process to
improve the teaching quality.
The educational process aims to transform the learning goals into learning outcomes
demonstrated by students. Learning outcomes are the nonempty set LO = {LO1, LO2,
…, LOn}, which should be clearly and unambiguously formulated in the course sylla-
bus. The teaching/learning process realizes the transformation.
The teaching process is based on three foundations: course content, delivery meth-
ods, and course materials. The learning process depends on students’ engagement and
involvement. The concrete realization of the education process depends on its form (for
example, full-time, part-time, e-learning). The process representation of education is
shown in Fig. 2.
Fig. 2. Scheme of the student-centered educational process
As the elements of LO describe the knowledge or skills students should acquire, they
focus on the context and potential applications of knowledge and skills. The assess-
ments and evaluations are the models of LOi, which provide possibilities to define the
level of students’ success. These assignments can take different forms, such as theoret-
ical research, case study, solving practical problems, testing, preparing presentations.
Due to the various types of assignments, they can be intended for individual or collec-
tive execution, have different time constraints, and allow or forbid the free choice of
performing tools.
Traditionally, the course professor develops the assignments and their evaluation
criteria according to the curricula. The evaluation scheme is specified in the syllabus.
Assignment grading can be organized with varying degrees of automation. In the case
of dual education, stakeholders can create and evaluate practical assignments as part of
real-life, ongoing projects. Also, students can perform self-assessment, passing inde-
pendent online testing on the course topics.
As it can be seen, despite а wide variety of assignment forms, they provide evalua-
tion only for observed teaching quality characteristics (OTQC). However, as it was
mentioned above, unobserved teaching quality characteristics (UTQC) have a signifi-
cant impact on the educational process as well. Therefore, the set of teaching quality
characteristics (TQC) should be considered as
TQC = OTQC ⋃ UTQC, (1)
where each element of the set corresponds to one or more measurers.
The vital component of the educational process is feedback [13]. It is necessary due
to several factors, for example, the influence of students on the quality of education,
the requirement of accreditation, the evaluation of the teaching staff of the educational
institution by the authorities of the educational establishment.
The educational process can be considered as a process with negative feedback. At
the same time, deviations of the actual values of TQC elements (TQCfact) from the rec-
ommended ones (TQCrec) should lead to formation of quality improvement activities
(QIA), which counteract the deviation to minimize it. In general, QIA can be represented
as an integral indicator of the impact on the educational process in the framework of
the chosen study program:
QIA = F (diff (TQCfact, TQCstd)) → min. (2)
The influence of OTQC on the quality of the educational process is well understood:
traditionally, assignments are evaluated on a quantitative scale using the existing rec-
ommendations. Therefore, we should study the set of UTQC elements, namely:
─ a set of unobserved teaching quality characteristics and their borderline values;
─ a set of possible measures, including scales and methods of assessment.
By this line of reasoning, the development and organization of quality improvement
activities cause a need of solving then following essential tasks:
─ the selection of UTQC and the definition of their borderline values;
─ the creation of effective mechanisms for assessing actual UTQC and their deviations
from the borderline;
─ the identification of the causes of deviations and analyzing the possibility of elimi-
nating them within a particular educational process;
─ the formation of quality improvement activities, taking into account the type and
degree of deviations for teaching quality improvement via implementing teaching
tactics.
UTQC can be determined indirectly by SET, while students provide feedback on the
course content, delivery methods, and course materials. Various techniques, such as a
questionnaire (preferred), survey, focus group, could be used to obtain the values of
UTQC characteristics. Usually the target properties of SET consist of
─ mindfulness: the awareness that course follows the syllabus, understanding of the
course role in the curriculum, satisfaction with their own progress in the course, ac-
ceptance of the course structure, clear presentation of the course material that facil-
itates understanding, clearness of the connections between the topics in the course
and with other courses of the curricula, understanding of the assessment methods;
─ adaptability: the correspondence of the course workload and requirements to the
course level, the acceptability of presentation speed;
─ efficiency: following the schedule, the time loss caused by access to the course ma-
terials, the timeliness of the information about schedule changes, aiding students’
learning by teaching methods, creation a welcoming and inclusive learning environ-
ment, timeliness of recommended reading and instructions, providing confidence to
do more advanced work in the subject, challenging and value of the course materials,
willingness, and ability of the professor to answer questions clearly and thoroughly,
returning assignments and exams on time, providing helpful feedback on time;
─ functionality: coherence of the requirements of the professor and teaching assistants,
the impact of instructional materials on increasing students’ knowledge and skills in
the subject, the availability of free and understandable reading in the library and
electronic access, updating and accuracy of the online course platform;
─ interference: knowledge or skills on the course subject earned early, the regularity
of class attendance, the effectiveness of organization and facilitation of learning ac-
tivities by the professor;
─ activity: preparation for classes, dealing with efforts aimed at doing reading tasks
and graded assignments.
When SET is over, it is advisable to analyze the characteristics of the evaluated foun-
dations of the teaching process:
UTQC = {UTQCCC, UTQCDM, UTQCCM}, (3)
where UTQCCC, UTQCDM, and UTQCCM are, respectively, the sets of unobserved char-
acteristics for the course content, delivery method, and course materials.
Formally, the unobserved characteristic utqc ϵ UTQC can be represented as:
utqc = , (4)
where Pr is one of the measurable properties listed above, TFnd is the evaluated foun-
dation of the teaching process (course content, delivery method, or course materials),
measure defines the set of measures for Pr.
The informative scales for the properties listed above and some examples for them
are shown in Table 1.
Table 1. Scales for Measuring Quality Characteristics
Type of scale Examples
Ratio scale ─ percentage of tasks performed without assistance
─ percentage of tasks completed on time
─ the number of missed/attended events (classes, consultations,
webinars)
Interval scale ─ percent range of tasks completed on time
─ percent range of attended classes
Ordinal scale ─ degree of satisfaction with the provided instructional materials
─ degree of the professor’s involvement and objectivity
─ degree of new material understanding
Dichotomous scale ─ the presence of prior knowledge on the course subject before the
course starts
─ submission of the completed assignment on time
The values of some properties can be measured by different scales. Thus, it is advisable
to select the most appropriate scales for measuring the values of the property. The scale
defines the method of value measuring and a corresponding method for statistical esti-
mation.
When conducting SET, a list of questions should be compiled; each question has to
correspond to one or more UTQC characteristics. The list of questions can be consid-
ered complete if it covers all valuable UTQC. The simplest way to achieve this is to
solve the problem of seeking the shortest coverage on condition that the list of questions
includes all valuable elements of UTQC.
5 The Means for Teaching Quality Improvement
5.1 The Formal Description of the Questionnaire for SET
Let us provide a formal description of the questionnaire as the most popular means of
feedback in the framework of the educational process.
There are various criteria for classifying questionnaires; for example, the number of
respondents, location, and the delivery method [14]. Online questionnaires have be-
come widespread due to their being easy to create, spread, and use. Faculties are using
survey platforms, such as Survey Maker or Survey Monkey [15], or the tools incorpo-
rated in the learning management system, such as Moodle [16].
Despite the simplicity and applicability of the questionnaire in all areas of life, there
are still some issues concerning the survey organization. Some of them are listed below.
1. Authorization during the survey
The advantage of anonymous surveys is a free expression of opinions. The disad-
vantages include the possibility of multiple filling-in of the questionnaire by one per-
son, the difficulty of maintaining anonymity in small groups, the challenge of control-
ling the completeness of the respondents’ group. The non-anonymous survey provides
such an additional advantage as a correlation with the parameters of the student’s pro-
file, in particular, academic performance, attendance, research work.
2. Time of the survey
The surveys are administered to students during or at the end of the course. The
advantage of the first option is the ability to correct the teaching performance for the
course presentation. The advantage of the second option is the lack of impact of the
professor’s impression on the evaluation of a particular student.
3. The universality of the questionnaire
The advantage of using a standard questionnaire for all courses is the possibility to
compare the results of the survey to create a rating of courses, programs, and professors.
The advantage of creating a specific questionnaire for each course consists in taking
into account the features of particular courses.
4. Assessment of a set of questions
It is necessary to consider whether the author of the questions is competent enough
from the professional expertise [17] and sociology points of view, whether the set of
questions is complete for estimation of teaching quality characteristics, whether the
questions and the measuring scales have been selected correctly.
5. The visibility of the results
It is necessary to determine who can see and analyze the results of the survey: the
course professor, the management staff, other teaching staff, students of the course,
other stakeholders.
6. The complexity of the interpretation and decision-making for quality improve-
ment activities in case of open-answer questions.
The problem of the questionnaire completeness remains an issue. As a possible so-
lution, we propose a formal description of the correspondence between the teach-
ing/learning process and the questionnaire.
The course teaching process is described as a set of foundations
TF = {CC, DM, CM}, (5)
where CC is the course content, DM is the set of course delivery methods, and CM is
the set of course materials.
The course content should be defined as a set; each element cciCC is defined as
follows:
cci = , (6)
where spi is the structural part (for example, lectures, visual presentation, command
project), qhi is the credit hours.
The delivery methods should be defined as a set; each element dmiDM is defined
as follows:
dmi = , (7)
where toi is the organizational component of the teaching process, and tpi is the descrip-
tion of the professor’s personality given by a set of characteristics tpiMTP ={Gender,
Age, Experience,...}. The set MTP can vary depending on the purpose of the question-
naire.
The course materials are also represented as a set; each element cmiCM is defined
as follows:
cmi = , (8)
where spi is the structural part which the material belongs to, kcmi describes the type of
presentation (for example, a printed form, a video, a presentation), vi defines the volume
of the course material, and ami defines the access method.
Each foundation of TF has its own set of characteristics to evaluate ССС NTQCCC,
СDM NTQCDM, СCM NTQCCM, respectively.
Now the questionnaire for SET can be described as follows:
SETQ = , (9)
where QN is the questionnaire used for the course TF in the yqn academic year.
QN is a set of questions, each of which is described as components:
qn = , (10)
where qtxt is the wording of the question text, kqn is the type of question, Cq’ is a set
of characteristics affected by the question qn (Cq’СССUСDMUСCM), ms is a scale of
measure.
The degree of completeness of the questionnaire QN is determined as the ratio:
QN
q 1
Cq'
QN . (11)
CCC CDM CCM
When QN=0, the questionnaire does not correspond to the course; when QN=1, the
questionnaire covers all fundamental properties of teaching quality characteristics. The
borderline value QN for a sufficient questionnaire could always be defined.
Let us describe the procedure of questionnaire development.
1. Form the set CCC as a collection of characteristics set for components of CC:
CCC CT SP
i 1 C C ,
spi qhi (12)
where CT is the set of characteristics for the course as a whole, SP is the set of structural
parts of the course, Cspi is the set of characteristics for the ith structural part of the
course, Cqhi is the set of characteristics for the credit hours of the ith structural part of
the course.
2. Form the set CDM as a collection of characteristics set for components of DM:
CDM Cto T
C
i 1 tpi , (13)
where T is the set of the course teachers (professor and teaching assistants), Cto is the
set of characteristics for the organizational component of the teaching process as a
whole, Ctpi is the set of characteristics for the ith course teacher.
3. Form the set CCM as a collection of characteristics set for components of CM:
C .
SP
CCM CM spi Ckcmi Cvi Cami (14)
i 1
4. Obtain the combined set C CCC CDM CCM .
5. Formulate the set of questions for сiС:
QNC qnC qtxtC , kqnC , ci , msC . (15)
6. Define the number of questions nq ci CQ
QNci , where CQC is the set of
characteristics with corresponding questions.
7. If CQ C , then calculate QN.
8. If QN QN
lim
, where QN
lim
is the borderline value, then return to step 5.
9. Find a sufficient set of questions.
The proposed procedure depends on the issue of sufficiency of the question set. Let
us suppose that the complexity of each question preparation is the same. The answering
efforts for all possible questions of the questionnaire are also the same. Then the defi-
nition of the questions optimal set of questions can be considered as the unweighted
problem of seeking the shortest coverage of the set of examined characteristics C with
the smallest set of questions from the set QNC.
The set of characteristics of C can be considered as a reference set С = {C1, …,Cn},
n=|C|. There is a set QN consisting of m subsets QNi of the set C, where QNi.CqC,
m
i 1
QNi .Cq C . The shortest coverage QN*QN is defined as
QN *
QNi .Cq C , QNi QN * , QN * min . (16)
i 1
This problem can be solved using the boundary-search algorithm on the concave set to
obtain all unabundant coverages. As a result, the shortest coverage QN*containing the
least number of questions is selected.
5.2 The Analysis of SET Results as Task of Informational Diagnostic
Because of the possibility to determine the borderline values for each quality charac-
teristic of the teaching process, processing of SET results to develop appropriate quality
improvement activities can be seen as the task of informational diagnostics. SET is a
rather complicated process; therefore, at the end of each iteration, it is necessary to fix
and determine the actual state based on the values of the characteristics [18]. A finite
set of diagnostic states allows us to differentiate SET results for all six properties of
every foundation of the teaching process. Besides, if several measurers with different
scales measure a property, then each scale provides its own set of diagnostic states.
Note that in the framework of this paper, we do not solve the problem of proving that
SET works on a normed Euclidean space, which is intuitively implied when using
measurers with quantitative values. We also do not unify different types of scales and
not provide recommendations on selecting scales for evaluating various properties of
the teaching process.
Let us consider the process of the formation of diagnostic states (classes) when using
various measures.
For the properties of the characteristics obtained by the measurers on the ordinal,
interval, and dichotomous scales, the appropriate classification method exists. The
number of classes is equal to the number of categories, the number of intervals, or two
(true/false), respectively. When using a ratio scale, it is advisable to bring the values
obtained on it to the interval scale. Diagnostics of SET results is performed using a
hierarchical classification method, which is characterized by the sequential division of
a set of objects (in our case, teaching process foundations) into smaller subsets (in our
case, properties of teaching process foundations).
We consider four levels of classification: the first level represents teaching process
foundations, the second level represents the properties of teaching process foundations,
the third level represents measurement scales, the fourth level represents sets of possi-
ble values. Diagnostic states are the vertices of the fourth level.
During SET, each question uniquely identifies the vertex of the third level. Each
answer to the question contributes to a fixed size (for example, a single one) to one of
the vertices of the subordinated fourth level.
In the normal state of the teaching process, the vast majority of contributions should
be concentrated at the vertices corresponding to the typical values of the measured
properties of the studied characteristics. For vertices that do not correspond to the typ-
ical values, the threshold values can be determined, exceeding of which signals about
the necessity of the additional analysis. Experts can evaluate these threshold values
based on the possible number of such vertices: the more vertices, the lower the thresh-
old. Cases of the threshold exceeding could be classified according to the conditions.
─ The threshold is exceeded for one vertex or adjacent vertices. The reason consists in
the poor quality of the teaching process conditioned by the poor quality of the corre-
sponding components of the teaching process. The required action is the formation
of quality improvement activities within the potential of technological means of the
resource-based approach.
─ The threshold is exceeded for several non-adjacent vertices (for example, for two
located at opposite ends of the scale). The possible reasons are the violation of SET
procedure or unaccounted teaching process characteristics that have a hidden effect.
The case requires additional research. The Catalog of Teaching Tactics
Another means created in the framework of the resource-based approach is teaching
tactics, which may be taken as fundamental or essential units of the professor’s behavior
helpful in creating a suitable learning structure for the realization of the set teaching-
learning objectives [19]. It is teaching methods, teaching techniques, teaching aid ma-
terials, and anything else helpful to them to realize their teaching objectives.
A teaching process usually bases on a collection of tactics. They have been used for
years, so now they are well isolated, cataloged, and described. To support professors,
we should accumulate the set of tactics formulated as “the diagnostic state – a tactic –
a result” as the core of the recommendation system. Three reasons cause the following
solution:
1. A professor can more easily assess the options for augmenting an existing process
to achieve a target value of the quality characteristic by understanding the role of
tactics.
2. Tactics give the professor insight into the properties of the resulting teaching activi-
ties.
3. By cataloging tactics, we provide a more systematic way of making a design of the
teaching process within some limitations.
The tactics usually overlap, and the professor frequently has a choice among multiple
tactics to improve a particular quality characteristic. The choice of which tactic to use
depends on such factors as tradeoffs among other quality characteristics, and the im-
plementation cost.
6 Conclusion
The work presents the research aimed at improving the teaching quality within the
framework of the resource-based approach. The resource development reflects the
model of multistage development, which means that each previous stage is the base for
all the following ones. In our case, the development of the teaching quality model
makes the first stage. The method formed at the second stage of resource development
is built on the base of the model. The means created at the third stage are information
technologies based on the model and encapsulated in the method.
At the first stage, we proposed a model for the teaching quality which reflects the
semantic and hierarchical relationship between
─ observed characteristics that can be measured directly;
─ unobserved characteristics that can be evaluated by analyzing students’ feedback;
─ measures that determine the teaching process characteristics quantitatively;
─ teaching tactics that describe all kinds of activities for the teaching process improve-
ment.
At the second stage, we proposed a method, which considers the teaching process as
a teaching quality management system with negative feedback. Measures of observed
and unobserved characteristics are compared with their threshold values. The difference
between the threshold and obtained values becomes the base for choosing the appropri-
ate teaching tactics. Each tactic describes the activities that lead to improving the par-
ticular quality characteristic and can affect three foundations of the teaching process,
namely the course content, delivery methods, and course materials.
At the third stage, we proposed a formalization of the questionnaire structure. Such
formalization supports covering and evaluating unobservable characteristics of the
teaching quality. In addition, the issue of teaching tactics cataloging is brought forth.
Such a catalog can support the systematic process of forming measures for teaching
quality improvement. This issue is the object of further research.
The comparison of SET results for different courses could provide such additional
benefits as:
─ determination of the appropriate sequence of courses in the curriculum from the stu-
dents’ point of view;
─ adjustment of the combination of structural elements in the courses of the curricu-
lum;
─ determination of the valid set of teaching aid materials and acceptable channels for
their delivery.
The proposed resource for improving the teaching quality does not depend on the
particular education form (such as full-time, part-time, e-learning). Comprehensive for-
malization could be realized at different automation levels of teaching quality improve-
ment caused by conditions and context of implementation. Continuous monitoring
gives the possibility to achieve and preserve the requested quality level of the teaching
process.
References
1. Thielsch, M., Brinkmöller, B., Forthmann, B.: Reasons for responding in student evaluation
of teaching. Studies in Educational Evaluation 56, 189–196 (2017). doi:
10.1016/j.stueduc.2017.11.008
2. Sauermann, J., Mengel, F., Zölitz, U.: Gender Bias in Teaching Evaluations. Journal of the
European Economic Association 17, 535–566 (2019). doi: 10.1093/jeea/jvx057
3. Drozd, J., Drozd, A., Antoshchuk, S., Kharchenko, V.: Natural Development of the Re-
sources in Design and Testing of the Computer Systems and their Components. In: Proceed-
ings of 7th IEEE International Conference on Intelligent Data Acquisition and Advanced
Computing Systems: Technology and Applications, 233–237, Berlin, Germany (2013). doi:
10.1109/IDAACS.2013.6662656
4. Schleicher, A.: Teaching and learning international survey TALIS 2018: Insights and Inter-
pretations. http://www.oecd.org/education/talis/
TALIS2018_insights_and_interpretations.pdf
5. Chen, W., Hendricks, K., Archibald, K.: Assessing pre-service teachers’ quality teaching
practices. Educational Research and Evaluation 17, 13–32 (2011). doi:
10.1080/13803611.2011.578008
6. Azam, M., Kingdon, G.: Assessing Teacher Quality in India. SSRN Electronic Journal
(2014). doi: 10.2139/ssrn.2512933
7. Dilshad, R.: Assessing Quality of Teacher Education: A Student Perspective. Pakistan Jour-
nal of Social Sciences 30(1), 85–97 (2010).
8. Reçi, E., Bollin, A.: The Evaluation of a Teaching Maturity Model in the Context of Uni-
versity Teaching. In: Proceedings of the 11th International Conference on Computer Sup-
ported Education – Volume 1: CSEDU, 360–367 (2019). doi: 10.5220/0007699303600367
9. Huber, S., Skedsmo, G.: Teacher evaluation – accountability and improving teaching prac-
tices. Educational Assessment, Evaluation and Accountability 28, 105–109 (2016). doi:
10.1007/s11092-016-9241-1
10. Ramlo, S.: Improving Student Evaluation of Teaching: Determining Multiple Perspectives
within a Course for Future Math Educators. Journal of Research in Education 27(1), 49–78
(2017).
11. Hornstein, H. A.: Student evaluations of teaching are an inadequate assessment tool for eval-
uating faculty performance. Cogent Education 4(1), 1304016 (2017).
12. Uttl, B., White, C. A., Gonzalez, D. W.: Meta-analysis of faculty’s teaching effectiveness:
Student evaluation of teaching ratings and student learning are not related. Studies in Edu-
cational Evaluation 54, 22–42 (2017). doi: 10.1016/j.stueduc.2016.08.007
13. McFadzien, N.: Why is effective feedback so critical in teaching and learning? Journal of
Initial Teacher Inquiry 1, 16–18 (2015).
14. Sokolova, М.: Methods of sociological research: textbook. Siberian state industrial Univer-
sity, Novokuznetsk (2008).
15. Marra, R. M., Bogue, B.: A critical assessment of online survey tools. In: Proceedings of the
Women in Engineering Programs and Advocates Network, 11th – 14th June, 2006. Pitts-
burgh, Pennsylvania (2006).
16. Büchner, A.: Moodle 3 Administration. Third Edition. Packt Publishing (2016).
17. Krisilov, V. A., Komleva, N. O.: Analysis and Evaluation of Competence of Information
Sources in Problems of Intellectual Data Processing. ProblemeleEnergeticiiRegionale 1-1
(40), 91–104 (2019). doi: 10.5281/zenodo.3239184
18. Komleva, N.: Some aspects of analyzing and diagnosing complex systems. In: Proceeding
of the International Multidisciplinary Conference on Science and Technology of the Present
Time: Priority Development Directions of Ukraine and Poland, Vol. 3, pp. 106–109,
Wolomin, Republic of Poland (2018).
19. Mangal, S. K., Mangal, Sh.: Learning and Teaching. PHI Learning Pvt. Ltd. (2019).