=Paper=
{{Paper
|id=Vol-2407/paper-12-121
|storemode=property
|title=
The process of automating the rating of Russian universities
|pdfUrl=https://ceur-ws.org/Vol-2407/paper-12-121.pdf
|volume=Vol-2407
|authors=Oxana N. Romashkova,Ludmila A. Ponomareva,Igor P. Vasilyuk
|dblpUrl=https://dblp.org/rec/conf/ittmm/RomashkovaPV19
}}
==
The process of automating the rating of Russian universities
==
https://ceur-ws.org/Vol-2407/paper-12-121.pdf
109
UDC 519.87
The process of automating the rating of Russian universities
Oxana N. Romashkova, Ludmila A. Ponomareva, Igor P. Vasilyuk
Department of Applied Informatics, Moscow City University
29 Sheremetevskaya str., Moscow, 127521, Russia
Email: ox-rom@yandex.ru, ponomarevala@bk.ru, ipvkod@mail.ru
The relevance of the study is determined by the need to create an automated system for
monitoring activities and determining rating indicators of universities in order to improve the
efficiency of the control and regulation of the functioning of the Russian universities.
The aim of the study: modeling the processes of determining the rating indicators of
universities and the formation of requirements for the information system for the center of
the rating assessment of universities.
Scientific novelty of research: modeling of processes of rating processing of universities,
formation and forecasting of rating estimates and on the basis of mathematical models
implemented in the intelligent modules of the information system, the ability to integrate the
system into the management information system of the University.
The authors conducted a study of information systems that carry out rating evaluation
on various indicators of higher education institutions. The functional requirements for the
developed information system for the rating evaluation center of higher education institutions
were formulated.
Key words and phrases: rating assessment, information system, University performance
indicators.
Copyright © 2019 for the individual papers by the papers’ authors. Copying permitted for private and
academic purposes. This volume is published and copyrighted by its editors.
In: K. E. Samouylov, L. A. Sevastianov, D. S. Kulyabov (eds.): Selected Papers of the IX Conference
“Information and Telecommunication Technologies and Mathematical Modeling of High-Tech Systems”,
Moscow, Russia, 19-Apr-2019, published at http://ceur-ws.org
110 ITTMM—2019
1. Introduction
In the post-industrial economy, the processes of knowledge generation, the formation
of intellectual capital and the introduction of the results of intellectual activity into
production are of particular importance. On this basis, one of the main subjects of the
economic system is the University as the main producer of knowledge. The University not
only affects the environment, but also itself is influenced. To determine the mechanisms
of mutual influence, it is necessary to try to explore the mechanisms of generation of
scientific knowledge. From this point of view, it is extremely difficult to analyze the
capital’s higher education institutions: on the one hand, they exploit the resources of the
entire country, on the other – affect the national economic system as a whole. Therefore,
the object of this study is selected regional universities, the relationship of which with
the local economic system is more obvious.
Higher education can contribute to the economy of the region in various ways: to
provide employment, increase consumer spending and revenues from outside the region
at the expense of students and teachers of the University. However, the most important
role of the University as a subject of the regional economic system is to generate
knowledge, which is an important resource in the digital economy. The popular model
of the University contribution was proposed by G. Itskovits. This model, known as the
triple helix, is based on the University – business – state partnership. In the future,
new components were added to the model: civil society and public institutions, then
the natural environment and environmental institutions. S. Hill developed a model
of involvement (engagement) of the University in the regional socio-economic system.
At the moment, a single methodological approach to the analysis of the impact of
regional higher education systems on economic growth has not been developed. However,
based on the analysis of the performance indicators of universities development it was
revealed that an important mechanism of interaction between the University and the
socio-economic system of the region is the unified rating system of the University.
The results of this work are important for future theoretical and applied research, as
well as for decision-making in the field of regional economic and educational policy.
In March 2013, the President of the Russian Federation instructed to develop a
national rating of Russian universities [1, 2]. Currently, more than 100 rating systems
are known in the world [3]. Out of those, around 20 are the most famous [4,5]. But all of
them solve problems characteristic of the national specifics of the country in which they
are created. In 2016, the first national Russian rating “three missions of universities”
was developed [6, 7].
The first analysis of the rating indicators was carried out in September 2017. 34
indicators were taken into account, combined into five groups: “Education” (40%
of the contribution to the result of the University in the ranking), “Science” (30%),
“Internationalization”, “Sustainability and development potential”, “Distance education”
(10%) [8]. According to the developed system, it is planned to rank not only Russian, but
also foreign universities — universities of Japan, China, Brazil, India, Iran, Turkey and
CIS countries. The developed evaluation system includes only measurable indicators [9,
10].
The creation of an information system for monitoring and evaluating the rating of
universities is part of the task of developing a national rating.
The object of the study are indicators and management processes for universities.
The subject of the research is the process of developing an information system for
the rating evaluation center of higher education institutions (IS REHEI).
2. Theoretical Study
Continuous monitoring of the state of affairs in the system of education and science
is necessary to compile the ranking of universities. Connections with both internal
elements of the system and with external objects should be implemented in the best
way [11, 12].
Romashkova O. N., Ponomareva L. A., Vasilyuk I. P. 111
Currently, there is no unified national information system for rating evaluation
of universities. But almost all universities have electronic educational environments,
management information systems, developed communication networks that implement
coordinated management of all structural units for the effective organization of the
educational process, conduct all types of training, scientific activity [13–15].
To analyze the level of automation of the rating assessment centers of higher educa-
tional institutions, five rating information systems were selected, presented below:
– “From the rating of Samara state University of Economics”;
– “Altai state technical University”;
– “IP rating estimation of activity of the academic units of the University of Economics
Vladivostok (VGUES)”;
– “Is planning and reporting of the work of VGUES teachers”;
– “And the rating of teachers of the Amur state University. Sholom Aleichem.”
The information system “is Samara state University of Economics” was developed by
the University staff to improve the efficiency of the educational process. The system is
based on the platforms ASP NET, MS SQL-server 2003. Because of this, its deployment
does not cause problems on almost any device.
Possibilities of information system “IC Rating of Samara state University of Econom-
ics”:
– calculation of individual student rating;
– automatic tracking of absenteeism and achievements;
– technology ADO.NET to work with database;
– easy system migration as there is no binding to the registry;
– calculation of the rating of the Department and teachers;
– calculation of the overall rating of automation of the University.
Lack of information system “IC Rating of Samara state University of Economics”:
– obsolete platform MS SQL-server 2003.
Information system “Altai state technical University (AltSTU)” was created by
teachers of Altai state technical University named after Polzunov. The system includes
two modules. One calculates the rating based on the potential-predicts how it will
change after the introduction of any changes. The second calculates the current rating.
The capabilities of information systems “Altai state technical University”:
– prediction of the rating taking into account the changes;
– monitoring the activity of students and teachers;
– personnel assessment;
– tracking publications;
– calculation of chairs rating
– calculation of the overall rating of scientific potential.
Disadvantages of information system "Altai state technical University»:
– no support for modern systems above Windows 7.
The IP rating estimation of activity of the academic units of the University of
Economics Vladivostok created by the students of this University, and used to control
the activities of the departments in it. The system uses a web interface and a MySQL-
based database.
Possibilities of information system “is of rating estimation of activity of educational
divisions of economic University of Vladivostok”:
– control of the Department;
– calculation of the rating of the Department;
– calculation of faculty rating;
– calculation of the rating of the student.
The lack of an information system of “IP rating estimation of activity of the academic
units of the University of Economics Vladivostok”:
– designed only for structural units and can not give a General picture of the state
of the University.
112 ITTMM—2019
Is planning and reporting work of teachers VGUES created for Vladivostok state
University of Economics and service. Allows you to monitor the activities of the
Department. The capabilities of information systems “is planning and reporting work of
the teachers of VSUES”:
– calculation of individual student rating;
– control of the Department;
– calculation of the rating of the Department.
The lack of an information system “is the planning and reporting work of the teachers
of VSUES”:
– functionality is limited only by the processes of the Department.
From the rating evaluation of teachers of the Amur state University. Sholom Aleichem
was created to control the activities of teachers of the Amur state University. Sholom-
Aleichem. The system has a Web interface.
Possibilities of information system “is evaluation of the rating of teachers of the Amur
state University. Sholom Aleichem”:
– calculation of individual rating of the teacher;
– calculation of the rating of the Department;
– monitoring of scientific achievements;
– calculation of rankings by year.
The lack of information system — control activities only at the level of the Depart-
ment.
As a result of the study of the subject area, the authors analyzed the information
systems of universities, which have the function of assessing at least one criterion included
in the formation of the rating of the educational organization: the rating of students,
departments, teachers, departments, etc. The results are presented in table 1.
On the basis of the analysis of information system, carrying out various ranking, the
conclusion was made about necessity of development of complex information system for
the rating assessment of the universities.
3. Problem Statement
Develop a model of functioning of is information system for the rating evaluation
center of higher education institutions, which should provide [16, 17]:
– loading of input data on educational institutions subject to analysis;
– the formation of a system of evaluation indicators and criteria support the changes
(increase in the number of indicators and benchmarks or to exclude them);
– adaptability to changes in the rating evaluation rules of higher educational institu-
tions;
– extension based on the performance results of a University by automating processes
missing;
– automated calculation of the rating of educational institutions based on the data
of the criterion assessment;
– interactive graphical representation of the rating calculation results for further
analytical work of the specialist. The authors have studied information flows that
occur during training [18–20].
4. The Functioning Model is Information System for the Rating
Evaluation Center of Higher Education Institutions
Development of the model of the processes of formation of the rating of the educational
organization was carried out with the help of the tool CA ERWin Process Modeler [21].
CA ERWin Process Modeler supports three modeling methodologies: IDEF0, IDEF3
and DFD [22].
For the study of the subject area was chosen methodology IDEF0 (Integration
Definition for Function Modeling), in which the business process is represented as a set
Romashkova O. N., Ponomareva L. A., Vasilyuk I. P. 113
Table 1
Comparison of Educational Process and Educational System [9]
From the Altai Is of a IC And from
rating of state rating es- plan- the rating
Samara tech- timation ning evaluation
state Uni- nical of activity and re- of teach-
versity OF Univer- of edu- porting ers of the
Econom- sity cational work Amur state
ics divisions of of the University.
economic teach- Sholom
University ers of Aleichem
of Vladi- VSUES
vostok
Calculation + + + + +
of stu-
dent’s
rating
The calcu- + + + – –
lation of
the rating
Group
The cal- + + + – –
culation
of the rat-
ing of the
Faculty
Calculation + + + + +
of the rat-
ing of the
Depart-
ment
Calculation + + + + +
of the rat-
ing of
teacher
Evaluation + + – – –
of technical
equipment
of elements-works that interact with each other, as well as showing information, human
and production resources consumed by each work.
Figure 1 shows the top-level context diagram of the business process model in IDEF0
notation.
Input interface arcs are: modernization plans, data on technical and information
equipment, performance reports, educational programs, data on publication activity,
data on personal achievements.
The output interface arcs are: a report on the rating assessment of the educational
organization, the rating assessment of the educational organization by indicators.
Administering the interface arcs are: Regulatory documentation and Federal educa-
tional standard, the Legislation of the Russian Federation [23, 24].
The mechanisms of implementation are: inspector, rating specialist.
114 ITTMM—2019
Figure 1. Context diagram top-level
Further decomposition of the central functional block was carried out. The resulting
A0 level decomposition diagram is shown in figure 2.
Figure 2. Diagram of the decomposition level A0
The study identified the following business processes: to assess the management of
the organization and content of education, to assess the teaching staff and teaching
methods, to assess the quality of knowledge and the independent work of students,
to assess the scientific effectiveness of the educational organization, to form a rating
assessment of the educational organization.
The function blocks described above are further decomposed. The most interesting
is the functional block “to form a rating assessment of educational organization”. Its
decomposition is shown in figure 3.
To automate the definition of rating evaluation, a diagram describing the sequence
of steps in the process of “Forming a rating assessment of an educational organization”,
performed in IDEF3 notation, was developed. It is shown in figure 4.
On the basis of the approved program, indicators for work are prepared, the rating
of students, the rating of teaching staff, the rating of equipment of the educational
organization is determined. Based on the final results, a decision is made to form a
rating assessment.
Romashkova O. N., Ponomareva L. A., Vasilyuk I. P. 115
Figure 3. Block decomposition Diagram “to form a rating assessment of an
educational organization”
Figure 4. Diagram of the description of the sequence of stages of the process “to
form a rating assessment of the educational organization”
On the basis of the performed analysis of the processes of activity of the center of
the rating estimation of higher education institutions the functional requirements to the
developed is were formed:
1. automate maintenance of the database of indicators of rating of the University;
2. determine the rating of University students;
3. determine the rating of structural units of the University;
4. determine the rating of teaching staff of the University;
5. determine the rating of University equipment;
6. form the rating assessment of the University on the main indicators;
7. report on the rating estimates of universities.
5. Conclusions
Functional requirements and model of the information system of automated moni-
toring of higher education institutions and definition of rating assessment, which has a
modular structure, which includes the following components: data collection module
116 ITTMM—2019
(data universities, individual structural units, branches, departments), classification
module, formalization and storage of data, the module determining the rating of the
University (generalized or individual indicators), data output module (reports, analytical
reports, statistical data). Each module acts as a separate independent subsystem and can
be integrated into the management information system of the University. All modules
are combined by a program shell, which is a window interface. Each module of the
system provides service utilities in the form of guidelines for use.
References
1. O. N. Romashkova, L. A. Ponomareva, I. P. Vasilyuk, Y. V. Gaidamaka, Application
of information technology for the analysis of the rating of university, in: CEUR
Workshop Proceedings 8. “ITTMM 2018 — Proceedings of the Selected Papers of
the 8th International Conference “Information and Telecommunication Technologies
and Mathematical Modeling of High-Tech Systems”, 2018, pp. 46–53.
2. E. Prokhorov, L. Ponomareva, E. Permyakov, M. Kumskov, Fuzzy classification and
fast rules for refusal in the qsar problem, Pattern Recognition and Image Analysis
(Advances in Mathematical Theory and Applications) 21 (3) (2011) 542–544.
3. M. Kumskov, E. Smolenskij, L. Ponomareva, D. Mityushev, N. Zefirov, The systems
of structure descriptors for solving the problems of “structure-activity” for organic
compounds, Reports of the Academy of Sciences 336 (1) (1994) 64–66.
4. O. Romashkova, T. Ermakova, Monitoring the quality of education in secondary
organizations with the use of modern means of Informatization, Bulletin of the
Russian University of friendship of peoples. Series: Informatization of education (4)
(2014) 10–17.
URL https://elibrary.ru/contents.asp?issueid=1345001
5. A. Samuylov, D. Moltchanov, Y. Gaidamaka, S. Andreev, Y. Koucheryavy, Random
Triangle: A Baseline Model for Interference Analysis in Heterogeneous Networks,
IEEE Transactions on Vehicular Technology 65 (8) (2016) 6778–6782. doi:10.1109/
TVT.2015.2481795.
6. A. Drozdova, A. I. Guseva, Modern Technologies of E-learning and its Evaluation
of Efficiency, Procedia — Social and Behavioral Sciences 237 (2017) 1032–1038.
7. V. S. Kireev, Development of fuzzy cognitive map for optimizing e-learning course,
Communications in Computer and Information Science 706 (2017) 47–56.
8. E. Ovchinnikova, S. Chiskidov, Problems of development and application of in-
teractive educational modules in learning, in: Science, education, society: trends
and perspectives the Collection of scientific works on materials of the International
scientific-practical conference: in 7 parts. OOO “Ar.Consult”., 2014, pp. 80–85.
9. L. Ponomareva, L. Kodanev, Development module of the corporate information
system "Educational environment of the University" based on cloud technologies, in:
Computer science: problems, methodology, technology the collection of materials of
XVII international scientific conference: in 5 t., 2017, pp. 393–398.
URL https://elibrary.ru/item.asp?id=28952199
10. F. Gasparetti, C. D. Medio, C. Limongelli, F. Sciarrone, M. Temperini, Prerequi-
sites between learning objects: Automatic extraction based on a machine learning
approach, Telematics and Informatics 35 (3) (2018) 595–610.
11. O. Romashkova, A. Morgunov, Information system for evaluation of results of
activity of educational institutions of Moscow, Bulletin of the Russian University of
friendship of peoples. Series: Informatization of education (3) (2015) 88–95.
URL https://elibrary.ru/item.asp?id=24323920
12. Y. Y. Orlov, D. Zenyuk, A. Samuylov, D. Moltchanov, Y. Gaidamaka, K. Samouylov,
S. Andreev, O. Romashkova, Time-dependent sir modeling for d2d communications
in indoor deployments, in: Proceedings — 31st European Conference on Modelling
and Simulation, ECMS., 2017, pp. 726–731.
13. E. Bobrikova, Y. Gaidamaka, O. Romashkova, The application of a fluid-based
model for the analysis of the distribution time of a file among users in peer-to-peer
Romashkova O. N., Ponomareva L. A., Vasilyuk I. P. 117
network, in: Selected Papers of the II International Scientific Conference “Con-
vergent Cognitive Information Technologies” (Convergent 2017). CEUR Workshop
Proceedings, Vol. 2064, 2017, pp. 55–61.
URL http://ceur-ws.org/Vol-2064/paper06.pdf
14. V. Borodakiy, K. Samouylov, I. Gudkova, E. Markova, Analyzing Mean Bit Rate
of Multicast Video Conference in LTE Network with Adaptive Radio Admission
Control Scheme, Journal of Mathematical Sciences (United States) 218 (3) (2016)
257–268.
15. I. Gudkova, O. N. Romashkova, V. Samoylov, Determination of the range of the
guaranteed radio communication in wireless telecommunication networks of IEEE
802.11 standard with the use of ping program, in: CEUR Workshop Proceedings
8. “ITTMM 2018 — Proceedings of the Selected Papers of the 8th International
Conference “Information and Telecommunication Technologies and Mathematical
Modeling of High-Tech Systems”, 2018, pp. 54–59.
16. O. N. Romashkova, S. V. Chiskidov, P. A. Frolov, Improvement of information
technologies of the decision of problems of management in ec.onomic systems,
Modern high technologies (10) (2017) 63–67.
17. O. N. Romashkova, L. A. Ponomareva, Model of educational process in high school
using Petri nets, Modern information technologies and it education 13 (2) (2017)
131–139. doi:10.25559/SITITO.2017.2.244.
18. O. N. Romashkova, L. A. Ponomareva, Model of effective management of the
United educational system (structure), in: New information technologies in scientific
researches materials of the XXI all-Russian scientific and technical conference of
students, young scientists and specialists. Ryazan state radio engineering University,
2017, pp. 16–18.
URL https://elibrary.ru/item.asp?id=30521101
19. V. Naumov, K. Samouylov, Analysis of multi-resource loss system with state-
dependent arrival and service rates, Probability in the Engineering and Informational
Sciences 31 (4) (2017) 413–419. doi:10.1017/S0269964817000079.
20. G. Gorelov, N. Kazanskii, O. Lukova, Communication quality assessment in speech
packet transmission networks with random service interrupts, Automatic Control
and Computer Sciences 27 (1) (1993) 62–64.
21. G. Gorelov, O. Romashkova, Influence of Russian, Spanish and Vietnamese speech
characteristicson digital information quality, in: IEEE International Symposium
on Industrial Electronics Proceedings of the IEEE International Symposium on
Industrial Electronics, ISIE’96. Part 1 (of 2). sponsors: IEEE, Warsaw University of
Technology. Warsaw, Poland, 1996, pp. 311–313. doi:10.1109/ISIE.1996.548438.
22. L. A. Ponomareva, V. Kodanev, S. V. Chiskidov, Model of management of process of
development of competences in educational organizations, in: New information tech-
nologies in scientific research materials of the XXII all-Russian scientific-technical
conference of students, young scientists and specialists. Ryazan state radio engineer-
ing University, 2017, pp. 20–22.
URL https://elibrary.ru/item.asp?id=30521104
23. L. A. Ponomareva, O. N. Romashkova, I. Vasilyuk, Conceptual model of changing
the rating assessment of the University, in: Methods, mechanisms and factors of
international competitiveness of national economic systems. Collection of articles of
the international scientific-practical conference: in 2 parts, 2017, pp. 75–77.
URL https://elibrary.ru/item.asp?id=30378981
24. L. A. Ponomareva, P. E. Golosov, A. B. Mosyagin, V. I. Gorelov, Method of effective
management of competence development processes in educational environments,
Modern science: actual problems of theory and practice. Series: Natural and
technical Sciences (9) (2017) 48–53.
URL https://elibrary.ru/item.asp?id=30281545