In the article the expediency of quality assessment of teaching the disciplines in higher education institutions was justified. A method of quality assessment level of teaching the disciplines and respective algorithm were proposed for making informed decisions in the quality management of education. This method is based on using the expert method of paired comparison and identifies two common forms of control: control at the level of teacher to learn the experience of colleagues (reciprocal visits) and a survey of students after studying disciplines. The paper concentrates on the correlation between indexes of teaching quality renders possible to quantify the overall index, which is called "level of teaching quality". Furthermore, the method takes into account the requirement of quality management systems to identify critical points and eliminate inconsistencies.
Rapid changes in the political, socio-economic and cultural realities in Ukraine and abroad, the current trends of globalization and integration of the world community have an impact on education. The modernization in education requires the use of innovative technologies, creative search of new strategic directions for content improving, methods, techniques and means of education and their application in the educational process of higher education institutions.
Higher education institutions begin to develop the quality management system at this
current stage of development of higher education in Ukraine, that meet the requirements
of ДСТУ ISO 9001:2015 (harmonized with ISO 9001:2015) [
In addition, significantly increases the role of modern information technology, namely the use of a single computer learning environment. This renders possible a systematic approach to raising the level of educational services within higher education institutions.
Therefore, raising the quality of educational services by application the assessment system of teaching subjects in a single information space is an important issue for higher education institutions. 2
The Application Relevance of Quality Assessment System of
Teaching the Disciplines
Today higher education in Ukraine is in the process of transformation from the Soviet
to the European model, established in accordance with the signed by Ukraine in 2002
Bologna agreement. By joining the Bologna process, Ukraine has pledged to work on
bringing the quality of national education in line with European standards.
The main focus in reforming of higher education made on the need to create the internal
quality systems. One of the main tasks of quality management systems according to the
standards requirements of ISO 9001:2015, which are implemented in modern
universities, is the presence of the feedback and determines the satisfaction degree of internal
customers – students. The education quality or obtaining specialists, which are required
by employers, primarily dependent on the quality of teacher’s work [
Analyzing the results and making decisions, based on them, is no less important stage of assessment quality of teaching the disciplines.
To ensure the quality of higher education have approved a number of normative documents in Ukraine such as [10,11]. They identified the main ways to improve the education quality and responsible departments.
In Sumy State University (SSU) the number of normative documents such as: Regulation "About the quality control system of the educational process in SSU", Regulation "About bureau of quality assurance of education and quality of higher education" and others were introduced [12,13,14,15]. These documents regulate the organizational forms and components of the quality control system of the educational process, monitoring the quality of educational activity and quality of higher education at all in its various components throughout the university. Also, the Council on ensuring the quality of educational activity and the quality of higher education was established at the university, which implements the university policy in the field of quality assurance and monitoring. Moreover, since 2015 the EU project Erasmus+ “Quality Assurance system in Ukraine: Development on the Base of ENQA Standards and Guidelines” (QUARE562013-EPP-1-2015-1-PL-EPPKA2-CBHE-SP) has been implementing in the SSU in cooperation with leading universities in Ukraine with the support of the Ministry of Education and Science of Ukraine.
But the analysis of requirements of normative documents of Ukraine and SSU has demonstrated that the criterias and methods of evaluating the quality of teaching are not identified, therefore the authors propose the development of a system and method of assessment of indexes of teaching quality.
It should be noted that nowadays the educational process is characterized by the introduction of modern information systems, which is to create single information space [16]. This greatly simplifies the exchange of information and data collection. In addition, IT renders possible create large amounts of the original data and analyze it as soon as possible.
SSU implemented a system of e-learning [17], which operates in a single information space of the university and consists of subsystems: storage and access to teaching materials; management of educational process; providing the development and filling of educational content; interaction of participants of the educational process and student’s electronic office. All subsystems are closely integrated with each other and work together in a single system.
So, the development of an assessment system of the quality of teaching the disciplines should be carried out with capabilities and requirements of information systems, which are implemented at the university. 3
Development of Assessment Method of Teaching Quality Indexes Analysis of national experience in the quality assessing of teaching the disciplines renders possible to identify two common forms of control: control at the level of teacher conducted to learn the experience of colleagues (reciprocal visits) and a survey of students at certain intervals of time or after studying particular disciplines. As already noted, the assessment criterias of the quality of the teacher of university are not defined normative. Therefore, in educational institutions there are different lists of requirements for teachers. Typically, they are used in the procedure of competitive selection for vacancies and assessment of these indexes is subjective and linguistic, for example "good" or "satisfactory".
To make decisions about the quality of teaching the disciplines it was proposed an overall index, which is called "level of quality". We should have an opportunity to evaluate it quantitatively for the proving our decisions. Therefore, the system of indexes of teaching quality was proposed (Fig. 1).
a ir e itr c n o i t a u l a v E
Contents classes For the assessing of the "level of teaching quality" of disciplines at the universities a comprehensive index of teaching quality is proposed, which is calculated according to the formula [18,19,20]:
Teaching quality indicators of engineering disciplines itllitlrrseaeeeaangvm tlrseeecuphu itlitanooy itltitirrsseaeeadhopon tsend M C Em onC tsu · · · ·
(1) The structure of classes Тeaching
Style
Personal skills
teacher and lf n se la r thP fso , e v i fan jtec ,n
o o ob ti rseeceephn ,ttircudoon liccaceognn ilscounn liiacepnw tfsecanko T in lo co oCm issau · · and ion s ,,litititlrracaayyqyu iitllittfrsaeaeybopn tiifseeacaohundg lC aav ehT · · n Qi і R i1
100 , % , 9 where Qi – value of unit quality index in classification group (in scores); n – number of indexes of teaching quality.
For clary presentation of the complex index, its value is normalized to a standard that has a value of 100%. Also has used a conversion factor of dimension – 100/9, as the maximum value of a single index of teaching quality that is equal to 9 scores. The assessment process is recommended to use according to the algorithm that is shown in Fig. 2.
In Fig. 2 the value of individual index of teaching quality is obtained through students in a questionnaire in the electronic office (block 3) or interviews with experts, teachers during reciprocal visits (block 4). To quantitative (scoring) characteristics of individual index of quality teaching Qi usually used scale, which is given in Table 1. Authors propose to substantiate values of the weighting coefficients of the index of teaching quality by carring out of using expert method of paired comparison – a set of logical, mathematical and statistical methods and procedures which related to the expert’s activities on processing information that is necessary for analysis and decision making.
The procedure for determining the weighting coefficients consists of two stages: the first stage is the selection of the optimal set of X features, the second is the set of numerical values of each feature X.
The coding factors and determination of expert’s opinion are carried out after work on the formation of the expert group. Results of analysis the opinions sheet of the member of expert group is given in the Table 2.
2
1. Incoming set of quality indicators; 2. Questionnaire for determining individual indicators Qi 3 Formation of the schedule of a survey of students
4
qualitative composition of experts 5
...
6 Processing of expert opinion. Determining the level of teaching quality R
9 7
No 8 Formation corrective action plan quality is shown by all criteria quality appears frequently, but not at all criteria quality appears in 50% criteria quality appears rarely and only on certain criteria no quality criteria for all The clarification of weighting coefficients in this work is proposed to realize by the method of successive approximation. In a Table 2 the values of weighting coefficients obtained by double paired comparison method of successive approximation, in which the result of measurement in the (w) approach is defined as the weighted mean. Initial results are considered in this case as a first approximation. 1 oTfhtehceocnltaesnst 1 2 1 2 1 1 8 0,222 46 0,229 257 0,231 2 dPirroefcetsisoinonal 0 1 1 1 0 2 5 0,139 28 0,139 152 0,137 3 lMogeitchaoldloe-vel 1 1 1 1 2 2 8 0,222 46 0,229 254 0,228 4 The structure of clas- 0 1 1 1 1 1 5 0,139 28 0,139 155 0,139 ses 5 sTteyaleching 1 2 0 1 1 0 5 0,139 28 0,139 158 0,142 6 Personal teacher 0 1 0 1 2 1 5 0,139 25 0,124 137 0,123 skills
Sum 36 1 201 1 1113 1 Note. The advantage of j-th object before i-th is numbered by 2, the equivalence by 1, and the advantage of j-th object before i-th is numbered by 0.
The second approach used as weighting coefficients γj(2) of experts opinions. Obtaining the new results with these weighting coefficients, in the third approach are considered again as the weighting coefficients γj(3) of the same expert opinion, etc. According to Theorem Peron-Frobenius [21], in particular conditions, this process converges, i.e weighting coefficients tend to some constant values that reflect the relationship between objects under examination established expert input.
The result of measurement of the j-th index in the first approximation Gj(1) (original result) determined by the formula:
m G j (1) K jl
l1 , where Kjl - number of benefits of index j by one expert (l=1…m ); Measurements of j-th index in (w) approaching equal [22]:
G j (w)
[(G1(w 1)]2 K j1 ... [(Gm (w 1)]2 K jm , where Gj(w-1) – outcome measurement of j index in a (w-1) approximation. The value of the weighting coefficients (w) approach defined as: j (w) m G j (w) j1 G j (w) (4) Values clarification process as long as the accuracy will not match the specified one, in other words, until reaching the condition: where – given the accuracy of calculations, which is accepted [20]: j (w) j (w 1) , 0, 001 при 1 a 1; , 0, 01 при a 5 (5) (6) where a – coefficient that shows: how many times the weighting of the best index exceeds the weighting of the worst index.
For the coefficient assessment of consistency of expert opinion, usually use the Kendal coefficient of concordance [22]. Consistency of expert opinions is considered as acceptable if the value of the coefficient of concordance W 0, 6 .
Summary table of weighting coefficients of indexes of teaching quality is the result of expert assessments (Table 3). The scale of desirability Harington is used for the formalization of complex quality index [23]. It refers to scales that are in the interval from zero to one. Therefore, for decision-making, based on comprehensive assessment "level of teaching quality", are offered the following recommendations (Table 4). 4
Practical Implementation of the Method of Assessment Teaching Quality The proposed in chapter 2 algorithm considers two forms of control: control at the level of teacher (reciprocal visits) and a survey of students at certain intervals of time or after studying particular disciplines.
Requirements for the first form of control are presented in SSU normative documents [12,13,14,15]. According to the requirements of these Regulations SSU introduced reciprocal visits. Annually forming composition of expert groups and schedule of visits to classes.
Recommendations for decision-making Very good quality. The process of teaching complies with all requirements; quality ensured.
Good quality. A minor mismatch. The process of teaching may be acceptable subject to the implementation of corrective action; there are insignificant inconsistencies (or one big inconsistency) Satisfactory quality. Much discrepancy. The process of teaching is satisfactory and can be conditionally enlisted; there are significant inconsistencies that shoul be eliminated. The process requires the development of analysis and eliminating of inconsistencies (or development of plan for corrective actions).
Bad quality. Critical discrepancy of teaching process. The process requires a time limit to eliminate inconsistencies.
Very poor quality. Complete mismatch. The process requires the development of a plan of corrective action, and time limit to removing inconsistencies and re-inspection.
However, the quality of teaching is not sufficiently appreciated without taking into account feedback from the students. The survey can be conducted through questionnaires or in the electronic office. SSU currently being tested "electronic personal office" – a single window access to various information services that will enable easy and timely receive customized information on training.
Practical implementation of the the proposed model of teaching quality assessment was conducted by the example of teachers of the department of Manufacturing Engineering, Machines and Tools. There were 32 students in specialty "Manufacturing Engineering" (level of Master of Science degree) on the five core disciplines teachers. The results data are summarized in Table 5. The teacher presence in the "satisfactory quality" explains his lack of experience teaching (first year after graduate school) and teaching a new course for the new curriculum. During the evaluation revealed the following critical points: the interconnection of material with professional interests and insufficient use of technical training aids.
Conclusions The work presents the research that aimed at improving the process of assessment of teaching quality at the universities, namely the current situation and ways of improvement of the teaching process were analyzed. The analysis of normative documents in field of education has shown the lack of a broad research of the question of assessing of teaching quality and the lack of uniform system of indexes of teaching quality. Therefore, as the result of research the overall index of teaching quality, the system of component indexes of quality and the method for its assessment in teaching the disciplines were proposed. In addition, the recommendations for decision-making based on the level of teaching quality were developed.
The application of the proposed method of assessment the teaching quality in universities in practice renders possible to: · on the basis of quantitative evaluation to carry out an objective assessment of the level of teaching quality at the universities; · make informed decisions in the quality management of educational process; · identify discrepancies (critical points) and criterias for developing a plan of corrective actions.
In the nearest future it is planned to improve normative base of higher education institutions through the development of a normative document, in which the requirements for teaching quality will be regulated and the method and nomenclature of indexes of assessment the teaching quality at higher education institutions will be presented. 10. Law of Ukraine "On Higher Education". URL: http://zakon2.rada.gov.ua/laws/show/155618/page. 11. Order of the Minister of Education and Science of Ukraine "Action Plan for pro-quality assurance of higher education in Ukraine and its integration into European and world educational community for the period up 2010”, № 612, July 13, 2007. URL: http://osvita.ua/legislation/Vishya_osvita/2514/. 12. Regulation "On quality control system of the educational process in Sumy State University".
Sumy State University, Sumy (2006). 13. Order "On creation of laboratory monitoring of quality educational activities". Sumy State
University, Sumy (2015). 14. Regulation "The quality assurance system of education and quality of higher education of
SSU". Sumy State University, Sumy (2016). 15. Order "On establishment of quality assurance and quality educational activities in education". Sumy State University, Sumy (2015). 16. Morzel, N., Kuzminska, O., Protsenko, G.: Public Information Environment of a Modern University. Proceedings of the ICTERI 2013, Kherson, Ukraine, June 19–22, 2013, CEURWS.org, 1000, 411–423. 17. The concept of building a unified educational e-learning environment at SSU.
URL: http://sumdu.edu.ua/ukr/general/normative-base.html. 18. Kysylevska, A.Yu., Nikipelova, O.N.: Qualimetric vipusku quality evaluation of medical treatments using water. Vestnik fyzyoterapyy and kurortolohyy, 2, 25–26 (2012). 19. Spivakovsky, O.V., Alforova, L.M.: Interpretation of assessing the quality of teacher received operation information service KSUFeedback. Information Technologies in Education, 24, 20–41 (2015). 20. Azhaldov, G.G.: Theory and Practice otsenki quality goods. Fundamentals of kvalymetryy.
Economics, Moscow (1982) [in Russian]. 21. Frobenius, G.: Ueber Matrizen Aus Nicht Negativen Elementen. Sitzungsberichte der Königlich Preussischen Akademie der Wissenschaften zu Berlin, 456–477 (1912). doi: 10.3931/e-rara-18865. 22. Azhaldov, G.G.: Estimates and Certification of quality in the construction. Stroiizdat, Moscow (1977) [in Russian]. 23. Harrington, E.C.: The Desirability Function. Industrial Quality Control, 21, 10, 494–498 (1965).