Modern information and communication technologies allow to successfully use the methods of effectiveness measurement of various processes in particular in the field of education. From our perspective, success in the implementation of individual educational trajectories by university students requires the assessment model based on the use of modern information technology. In this article, we propose our own methodology for such measuring. It takes into account the entrant's competitive score and student's rating score. The user of the created system can choose the necessary information by criteria such as higher education degree, the form of study, department, and using the graph visualize the implementation of each student's educational trajectory. The presented methodology allows for evaluating objectively the quality of students` individual study plans and can be used in the development of universities rating and ranking criteria.
A substantial reorganization of university education is taking place in Ukraine at the
present time. An example of this process is the establishment of the National Agency for
Higher Education Quality Assurance in 2015. It is a permanent collegial body
empowered to implement state policy in this area. Its powers include the following:
development of the standards and criteria of higher education quality assurance,
development of the procedure for accreditation of higher education institutions; analysis
of the universities activities quality; conducting a licensing examination; preparation of
an expert opinion on the possibility of issuing a license to carry out educational activities
Copyright © 2020 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
in the field of higher education; formation of criteria for quality assessment of higher
education institutions activities on which ratings of these institutions be compiled etc
[
In recent years, the issue of the quality of higher education in Ukraine has been the
subject of a number of papers, in particular in the OECD “Reviews of Integrity in
Education: Ukraine 2017” [
For example, at Lithuanian Sports University they decided to find criteria of education quality assessment with a help of questionnaire which was given to students. Such quality assurance criteria were investigated: 23 factors that could determine education quality; 33 statement about role of teaching staff and its influence; 35 methods that could determine material quality on lectures; 14 methods with accessible learning resources for subjects.
In conclusion they found that contents of study, form of curriculum, teaching stuff
achievements, the level of funding as well as infrastructure, using IT systems and public
information were the most important criterias in studying process [
Individual monitoring of German universities quality assessment presents individual achievements of each student separately as well as each teacher. Intra-university assessment of education quality includes such criterias as: separate assessment of each student or teacher achievements; teachers monitor the students achievements during the educational process by themselves; administrative assessment is done by the administration of university in the system direction and control [13].
Issued by the Norwegian Agency for Quality Assurance in Education criteria for analysis of quality assurance systems shall involve the entire establishment and shall apply to the square measures of activity that are associated with quality of studies and also the total learning surroundings for all course provisions that the establishment is accountable, each internal and external. In evaluating the standard assurance system, stress is placed on the subsequent aspects of the system.
Active participation by students in work on quality and also the concentrate on the overall learning environment. An annual report on work on quality to the board of the establishment, giving a coherent overall assessment of quality of studies at the establishment and an summary after all plans and measures for work on quality [14].
How the college rankings were сalculated in USA.
These square measure the factors and weights utilized in the U.S. News Best schools rankings.
They approach outcomes (35%) from angles of graduation and retention (22%), graduation rate performance (8%) and social quality (5%).
Graduation rate performance: they compared every college's actual six-year graduation rate with what we tend to foretold for its fall 2012 coming into category. the anticipated rates were sculptural from admissions information, proportion of undergraduates awarded Pell Grants, faculty money resources, proportion of federal help recipients World Health Organization square measure 1st generation, and National Universities' maths and science, or STEM, orientations.
Social mobility: Measures however well colleges graduated students World Health Organization received federal Pell Grants (those generally coming back from households whose family incomes square measure but $50,000 annually, although most Pell Grant cash goes to students with a complete family financial gain below $20,000).
They conjointly aforementioned however the graduation and retention rate benchmark ranking was calculated. it's all over again computed from a school's total score in these 2 ranking indicators: average six-year graduation rate and average freshman retention rate [15].
Faculty Resources (20%)
Research shows the bigger access students ought to quality instructors, the a lot of engaged they'll be in school and therefore the a lot of they'll learn and sure graduate. U.S. News uses 5 factors from the 2018-2019 year to assess a school's commitment to instruction: category size, college regular payment, college with the very best degree in their fields, student-faculty magnitude relation and proportion of school World Health Organization square measure full time.
Expert Opinion (20%)
Academic name matters as a result of it factors things that can't simply be captured elsewhere. as an example, an establishment well-known for having innovative approaches to teaching could perform particularly well on this indicator, whereas a college troubled to stay its certification can probably perform poorly.
Financial Resources (10%)
Generous per-student defrayment indicates that a university offers a large style of programs and services. U.S. News measures money resources by victimization the typical defrayment per student on instruction, research, student services and connected academic expenditures within the 2017 and 2018 business enterprise years. defrayment on sports, dorms and hospitals doesn't count.
Student Excellence (10%)
A school's educational atmosphere is influenced by the property of its admissions. Simply put, students World Health Organization achieved sturdy grades and take a look at scores throughout high school have the very best likelihood of succeeding at difficult college-level coursework; sanctioning instructors to style categories that have nice rigor.
Alumni Giving (5%)
This is the typical share of living alumni with bachelor's degrees World Health Organization gave to their faculty throughout 2016-2017 and 2017-2018. Giving measures student satisfaction and post-graduate engagement. 2
The purpose of the development of our methodology is measuring the effectiveness of
the implementation of individual educational trajectories by university students using
modern information and communication technologies. The starting point of
measurement is the indicators with which the applicants are enrolled in a higher
education institution [
to the first year for gaining Bachelor’s Degree (Master’s Degree in the field of medicine) based on the complete general secondary education
Contents of the component
Is the form of the entrance test that involves assessing the preparedness and motivation of the applicant. Based on its results, a protocol decision is made regarding providing an applicant with a recommendation for admission.
Is a comprehensive assessment of the applicant's achievements. The calculation based on the results of entrance tests and other indicators up to 0,001 in accordance with the Entrance Terms and Conditions.
Competitive score (CS) = C1*S1 + C2*S2 + C3*S3 + K4*A + C5* OU, where C1, C2, C3, C4, C5 are integral weighting coefficients. Their sum for each competition proposal must be equal to 1;
S1, S2 - External independent testing (EIT) scores or entrance exams scores of the first and second subjects;
S3 - External independent testing (EIT) scores or entrance exams scores of the third subjects and creative competition;
A - the average grade of the document on the complete secondary education;
OU - a score for the successful completion of the KSU preparatory courses of the specialty, which is given special support.
Finally, the competitive score is multiplied by the product of the following factors: AC, IC, RC, PC.
The maximum competitive score may not exceed the figures 200.
a) for entrance for the Master's degree in specialty in specialty 081 "Law", in other specialties (except the specialties of branches of knowledge 01 "Education / Pedagogy", specialties 025 "Musical art") according to the formula: Competitive score (CS) = P1 +P2 + P3, where the components take into account the specifics of certain specialties;
b) in other cases, the competitive score is calculated as the sum of the entrance Rating score exam scores and the average score of the supplement to the document on preliminary education, taking into account the right to the primary enrollment (by multiplying the competition score by 1.05) to higher medical and pedagogical institutions.
Determines a person's place in the ranking for an academic scholarship.
Consists of average success score and additional scores.
The average success score of student’s academic achievements according to the results of semester control is determined on a 100 point scale of assessment. It is 95% of the rating score. Additional scores are 5% (4.75 scores). They are calculated according to the following indicators:
a) achievements in scientific, scientific and technical activities (2.5%
additional scores, maximum - 2,375 scores);
b) active participation in international programs, projects, seminars, forums, competitions (1% additional scores, maximum - 0.95 scores);
c) active participation in public life (1.5% additional scores, maximum - 1,425 scores).
It follows that the maximum student rating score - 104.75 scores (100 scores is the maximum success score + 4.75 is the maximum number of additional scores).
So, we get the initial information from website Vstup.Info. With the help of the parser developed by us, we collect entrants' competitive scores. If a person is enrolled in university, we enter his name and score in the database.
At the same time, we turn the competitive score on a 200-point scale into a 100-point scale in advance. The results of the session a Dean`s Office submits in xml file format. The administrator of this platform downloads student success data. According to the results of the semester exams, it is the main information for assessing and rating a student.
As can be seen from the Table 2, the rating score includes the average study score and
other scores, including scores for the participation in scientific activities. As the student
participates in the scientific activity of the department and faculty, the participation in
conferences, publication of articles and abstracts are taken into account. In the articles
[
After passing the exams, the rating examiner checks whether the student has a publications. This can be done through the KSU Publication service. The list of articles written by student K. Panova is given in Table 3.
scientometric indices of universities Information system of scientific activity indicators of scientific organizations: Development status and prospects + + + + + + This system displays a grah indicating the connections between the authors.
Fig. 1. Connections between co-authors of student K. Panova The model of data representation in the rating system is presented in Figure 2: 1. Assessment is the entity, will include such entities as the subject and the student. 2. Student is the entity, will be used as the external key in the entity Assessment. 3. The subject is the entity will be used as the external key in the entity Assessment. 4. Specialty is the entity, will be used as the external key of the Student entity and contains the Faculty entity. 5. Faculty is the entity, will be used as the external key of the Specialty entity. For now, we will analyze the data of only one faculty. In the future, we assume the possibility of the developing of data on several other faculties.
The "administrator" of the system can take such actions: ● Add (delete) the following entities: ─ student, ─ discipline, ─ specialty, ─ rating, ─ faculty. ● Filter by: ● Search by: ─ name, ─ specialty, ─ discipline, ─ rating. ─ form of study, ─ higher education degree, ─ specialty.
It is possible to visualize these requirements with the help of use-case diagrams (Figure 3). Based on the previous actions we obtain the result presented in Figure 4:
As can be seen from Figure 5, the system user is able to select the required information by degree, form of study, department, as well as visualize the implementation of the educational trajectory of each student using a graph. Figure 6 shows how filters can determine when a bachelor's student completed the educational trajectory most effectively and vice versa. Below is a visualization of the implementation of educational trajectories by Master` Degree students (Figure 6):
It was concluded that the proposed model allows measuring the efficiency of the implementation of individual educational trajectories by university students based on higher education degree, a form of study and the department. The presented method also allows objective evaluating the quality of individual curricula plan of students and can be used in developing criteria for ranking and rating of universities.
It is suggested that based on the trajectories of student success, using the methods and tools of machine learning (ML), the demonstration of malicious behavior of participants in the educational process can be tracked.
By malicious behavior we mean illegal behavior of a student and / or scientific and pedagogical staff, which is manifested in the following violations of academic integrity as defined by Article 42 of the Law of Ukraine "On Education": • corrupt practices is the provision (reception) by a participant of the educational process or a proposal to provide (receive) funds, property, services, benefits or any other benefits of a material or intangible nature in order to obtain an undue advantage in the educational process; • biased assessment - deliberate overestimation or underestimation of the assessment of learning outcomes of students; • providing assistance to students during their assessment of learning outcomes or creating obstacles that are not provided by the conditions and / or procedures for such assessment.
The following trajectories can be considered as patterns of criminal behavior that are indicators: 1. The student(s) has a volatile curve, the average score constantly fluctuates from low to high and from high to low. 2. The student(s) throughout the period of study has / have a low average score, and at the certification receive a high.
These patterns may indicate bribery, biased assessment, and the provision of learning outcomes to learners during their assessment that are not covered by the conditions and / or procedures for such assessment.
These patterns may indicate corrupt practices, biased assessment, and the provision of learning outcomes to learners during their assessment that are not covered by the conditions and / or procedures for such assessment.
3. The student has a high average score throughout the study and at the certification receives a low score.
This result may indicate a violation of the student's right to a fair and objective assessment of learning outcomes due to the biased attitude of the researcher to the student.
Using the proposed approach, the algorithm can be trained in the first three cases and trace other malicious behavior, which is a combination of the three options described above. We also emphasize that the grounds for raising the issue of establishing the facts of academic dishonesty of participants in the educational process, in our opinion, can only be systemic cases, individual ones should be considered as an error.
Prospects for further research are to develop a model for measuring the effectiveness of the implementation of individual educational trajectories by university students which will include success indicators of graduate student, such as employment, salary etc. Higher Education in Tourism and Sport Management Study Programme. 14. Criteria for evaluation of universities’ and university colleges’ quality assurance systems for educational activities (issued by the Norwegian Agency for Quality