The article is directed to the development of an information-analytical system for evaluating the activity of scientific-pedagogical staff based on KPIs at the Odesa National Maritime University. To realize the goal, the mission of Odesa National Maritime University was defined, the mechanisms of project management of the university were developed based on key performance indicators, the algorithm for the functioning of the information-analytical system at the university was determined, and the technical task for the information-analytical system for evaluating the activity of scientificpedagogical staff was developed based on KPIs at Odesa National Maritime University. It was proposed to include basic and additional KPIs in the database of the information system. The development of an information-analytical system for evaluating the activities of the scientificpedagogical staff of the university corresponds to the main directions of the development of the university, namely, the direction of informatization, management of the institution of higher education and personnel potential, improvement of the educational process, increasing the efficiency of scientific research and innovation. The result of the implementation of the information-analytical system for evaluating the activity of scientific-pedagogical staff based on KPIs is the improvement of the efficiency of the scientific-pedagogical work of university employees and the expansion of the scope of the use of information technologies and the project approach in the management of a higher education institution.
Institutions of higher education occupy a significant place in the development of the state by providing the knowledge, skills, ideas, fundamental and applied scientific research that any country needs to ensure economic, social, political development and growth. Certified by the International ISO 9001:2008 standard, the unique Odesa National Maritime University (ONMU) actively implements progressive methods of training modern specialists for the needs of Ukraine as a maritime state and foreign countries.
The influence of Odessa National Maritime University on the future ensures the formation of a new corporate culture, the use of advanced digitalization technologies and the improvement of interaction skills within the organization, which determines the development of the university as a center of advanced ideas, which is integrated into the world educational and scientific space.
One of the main directions of development of ONMU is the use in its activities of information-analytical mechanisms of project-oriented management of the activities of the university based on key performance indicators to ensure high-ranking positions among the list of leading universities of Ukraine and improve rating indicators in global ranking systems institutions of higher education.
The development and implementation of the university development plan based on the development of an information-analytical system for evaluating the activities of scientificpedagogical staff based on key performance indicators will allow to focus efforts and optimize the allocation of resources for the implementation of measures necessary to fulfill the mission of the University and ensure its development in conditions of growing competition.
Institutions of higher education are an important component of the development of both a
separate region and the country as a whole. However, the relationship between ‘town and gown’
has often been criticized as not always being aligned, limiting the potential economic and
societal benefits a close relationship might bring to the city and region [
Analytical studies of international and Ukrainian experience in using advanced information
technologies in university management are presented in works [
In [
Analyzing the risks that arose when martial law was introduced in Ukraine, the work [
Drawing on the resource-based view, the study [
The study [
The presented research makes it possible to establish the relationship between key
performance indicators by analyzing the features of the influence of indicators on the
performance of a higher education institution. This approach allows you to select the most
suitable performance indicators for the implementation of the university’s strategic plan based
on the use of a model for managing these indicators. Work [
In [
A linearized model for academic staff assignment in the university focusing on performance
gain in quality indicators is developed in [
Studies [
A multi-criteria approach to systems management, as well as models for solving optimization problems of systems development, are presented in research papers [32-34].
Other issues of modeling information systems and developing technical specifications for them, taking into account the specifics of the activities of organizations, have been studied in research [35-40].
This article is aimed at achieving the following goal – the development of an informationanalytical system for evaluating the activities of scientific-pedagogical staff based on KPIs at Odessa National Maritime University.
To realize the goal of the research, the following tasks were set: defining the mission of ONMU, developing the mechanisms of project-oriented management of a higher education institution based on key performance indicators, determining the algorithm for the functioning of the information-analytical system at the university, developing the technical task for the information-analytical evaluation system activities of scientific-pedagogical staff based on KPIs at ONMU.
The mission of ONMU is to train specialists who are competitive in the global labor market in the field of water transport, by attracting leading scientific and scientific-pedagogical staff, using the results of scientific research and implementing innovative teaching methods.
To implement the mission, the university defines six main areas of development, which are interrelated and for the implementation of tasks, the interaction of the administration, structural divisions, scientific-pedagogical staff and students of higher education, as well as their public associations is necessary.
The creation of an information-analytical system for evaluating the activities of the scientific-pedagogical staff of the university corresponds to the main directions of the development of the university, namely, the direction of informatization, management of the institution of higher education and personnel potential, improvement of the educational process, increasing the efficiency of scientific research and innovation.
• • • • • • • • • • • •
The mission of the development of an information-analytical system for evaluating the activity of scientific-pedagogical staff based on KPIs at ONMU using the Moodle information system platform is to ensure the quality of the composition of scientific-pedagogical staff by the use of mechanisms for evaluating and self-evaluating the effectiveness of scientific-pedagogical activity, its direction according to development priorities of the national system of higher education, development strategy of ONMU, personal priorities of the professional activity of scientific-pedagogical staff.
The maximum assessment can be achieved only under the condition of constant, diverse, intensive activity of scientific-pedagogical staff, which stimulates their personal and professional development and is aimed at increasing the level of indicators of the development of departments, faculties, educational and scientific institutes of the university in general.
The purpose of the development of an information-analytical mechanism for evaluating the activity of scientific-pedagogical staff based on KPIs at ONMU is:
Increasing the efficiency of the professional activity of scientific-pedagogical staff Ensuring the objectivity of the evaluation of the activities of the scientific-pedagogical staff of the university based on the use of the information system of data accumulation Ensuring competition, increasing motivation and work efficiency Improving the quality of higher education and complying with ONMU's performance indicators with licensing and accreditation conditions and establishing compliance of the teaching staff with licensing requirements for educational activities Accumulation of data in the Moodle information system platform to identify the dynamics of the departments and educational and scientific institutes of ONMU Stimulating the activities of scientific-pedagogical staff aimed at improving the quality of education
The main tasks of the development of an information-analytical mechanism for evaluating the activity of scientific-pedagogical staff based on KPIs at ONMU are:
Creation of an information base for the analysis and evaluation of the results of the activities of scientific-pedagogical staff based on KPIs at ONMU, including when filling vacant positions of scientific-pedagogical staff Increasing the interest of scientific-pedagogical staff of the university in improving their professional qualifications Ensuring the objectivity of assessments of the quality of the activities of the scientificpedagogical staff of ONMU due to the completeness and reliability of the information uploaded to the Moodle information system Activation and stimulation of activities that orient and contribute to increasing the rating of ONMU and its development in general by the set strategic priorities, creating conditions for the professional growth of scientific-pedagogical staff of the university Improvement of the system of stimulation of the scientific-pedagogical staff of ONMU Implementation of information technologies in university activities
The technical task for the information-analytical system for evaluating the activity of scientific-pedagogical staff based on KPIs at ONMU is based on the following components: • • •
Correspondence of the content of the assessment to the priority areas of development of ONMU Objectivity and reliability of the information obtained from the information-analytical system for evaluating the activity of scientific-pedagogical staff Stimulation of scientific-pedagogical staff of ONMU to improve the results of their professional activities
The implementation of KPIs-based assessment of the scientific-pedagogical staff at ONMU is an integral element of the development of a monitoring system and the use of modern information technologies as a component of the process of ensuring and managing the quality of higher education, the internal system of ensuring the quality of higher education of ONMU, stimulating the improvement of qualifications, professionalism, productivity of educational and scientific work of scientific-pedagogical staff.
The main requirements for the technical task of the information-analytical system for evaluating the activity of scientific-pedagogical staff based on KPIs at ONMU are an objectively determined number of indicators entered into the information-analytical system that characterize the professional activity of scientific-pedagogical staff in terms of achieving the goals set by the Development Strategy of ONMU and assessment of compliance of the activities of the scientific-pedagogical staff of ONMU with job qualification requirements and job instructions, Licensing conditions for conducting educational activities, as well as the importance of the personal contribution of scientific-pedagogical staff in solving the tasks of the department, faculty, educational and scientific institute of the university.
The model for the formation of a technical task for the development of an informationanalytical system for evaluating the activity of scientific-pedagogical staff based on KPIs at ONMU looks as follows (Figure 1).
The organization of work to support the KPIs-based evaluation system of scientificpedagogical staff at ONMU is carried out by the person authorized to implement the KPIs system at ONMU, who is appointed by the order of the Rector.
The evaluation of the activity of scientific-pedagogical staff (SPS) based on KPIs at ONMU as part of the information-analytical system is carried out by the Rector according to the basic and additional obligations listed in Table 1 and Table 2.
The types of basic and additional obligations that form the technical task of the informationanalytical evaluation system are determined by the Academic Council and put into effect by the order of the Rector. Data for technical specifications for the development of an information system on the basic and additional key performance indicators for scientific-pedagogical staff of ONMU are presented in Table 1 and Table 2.
LIST OF MAIN AND ADDITIONAL KPIs
Scientific-pedagogical staff have the right to choose additional obligations listed in Table 2 to increase their professional level, as well as to increase the Rector's assessment of the activity of a scientific-pedagogical worker based on key performance indicators.
The academic year is taken as the reporting period. Based on the results of the reporting period, the scientific-pedagogical staff of ONMU prepare a report on the achievement of the basic KPIs and additional KPIs. In the report, the scientific-pedagogical staff of ONMU conducts a self-assessment of the results of their activities for the reporting period with information on the achieved indicators.
The report is signed by the executor with an indication of the date and approved by the head of the department. The report of each scientific-pedagogical worker is considered at the department meeting until June 10 of the reporting period.
Based on the results of consideration at the department meeting of the reports of all scientific-pedagogical employees of the department, an extract from the protocol of the department meeting is drawn up with a decision on the assessment of the effectiveness of the work of each scientific-pedagogical employee of the department for the reporting period.
The effectiveness of the work is confirmed by the performance of the basic KPI and additional KPI by the scientific-pedagogical staff of the department during the reporting period.
KPI – 1. At least one publication from points 1.1 or 1.2: Publication of scientific articles in Ukrainian specialized journals Publication of scientific articles in periodical scientific journals
(Scopus, Web of Science) KPI – 2. Participation in international and regional conferences
with the publication of theses KPI – 3. The presence of a positive student assessment of the quality of teaching of an academic discipline according to a
questionnaire of higher education applicants KPI – 4. Having/obtaining a scientific degree and/or academic title
KPI item
KPI – 1. Publication of a textbook, study guide, including № 1 1.1 1.2 2 3 4 № Condition for KPI
acceptance Individual article / Co-authored article Individual article / Co-authored article Individual report / Co-authored report those published in co-authorship (with a volume of at least 1.5
author sheets (a. s.) for each co-author) KPI – 2. Publication of monographs (with a total volume of at least 5 a. s.), including those published in co-authorship (with
a volume of at least 1.5 a. s. for each co-author) KPI – 3. Participation as an official opponent or a member of a permanent specialized academic council, or a member of a
one-time specialized academic councils KPI – 4. Participation as a scientific supervisor or responsible executor of scientific projects, or a member of the editorial board/reviewer of Ukrainian or foreign scientific journal
indexed in bibliographic databases KPI - 5. Involvement of students in participation in the All
Ukrainian Student Olympiad KPI - 6. Involvement of students in participation in the All
Ukrainian competition of student scientific works KPI – 7. Participation of scientific-pedagogical staff in the organizational committee/jury of the All-Ukrainian
competition of student scientific works KPI – 8. Management of a student scientific, problem group
KPI – 9. Participation of scientific-pedagogical staff in
professional and/or public associations by specialty KPI-10. Performing the functions of the guarantor of the
educational program KPI-11. Teaching professional subjects in English (KPI-11 is not accepted for scientific-pedagogical staff of the Department of Philology) or in co-authorship
Individually or in co-authorship
The department prepares a general departmental report on the evaluation of the activity of a scientific-pedagogical employee of the department based on KPIs, which consists of reports of all scientific-pedagogical employees of the relevant department, which must be signed by the executors and approved by the head of the department.
By June 15 of the reporting period, the department uploads the following documents to the Moodle information system platform in the course category "Evaluation of the activity of scientific-pedagogical staff based on KPIs at ONMU", choosing the appropriate department: • •
Extract from the protocol of the meeting of the department General departmental report on the evaluation of the activities of scientific-pedagogical employees of the department based on KPIs
Heads of departments control the timeliness of uploading general departmental reports on the evaluation of the activity of scientific-pedagogical employees of the department based on KPIs into the Moodle information system platform.
Heads of departments and scientific-pedagogical staff of ONMU are personally responsible for the reliability of the information provided in the report of the scientific-pedagogical staff and the general departmental report on the evaluation of the activities of the scientific-pedagogical staff of the department based on KPIs. The authorized person for the implementation of the KPIs system at ONMU reports to the Rector on the timeliness of uploading reporting documents by departments.
The evaluation of the activity of scientific-pedagogical staff based on KPIs at ONMU is carried out by the Rector based on reports posted in the Moodle information system platform based on the results of the reporting period.
Information on the evaluation of the activity of scientific-pedagogical staff based on KPIs at ONMU is used for differentiated evaluation of the activity of each scientific-pedagogical worker, during competitive selection, etc.
The KPIs-based evaluation information-analytical system of scientific-pedagogical staff at ONMU is used to summarize the activities of scientific-pedagogical staff for the Rector to make personnel decisions and provide recommendations to scientific-pedagogical staff to improve their work.
The results of the evaluation of the activity of scientific-pedagogical staff based on KPIs at ONMU are used when making decisions regarding moral and material encouragement of scientific-pedagogical staff of ONMU, and competitive selection to fill vacant positions of scientific-pedagogical staff.
The system of evaluating the activity of scientific-pedagogical staff based on key performance indicators at ONMU is not applied to the administrative and managerial staff appointed to the position of scientific-pedagogical worker. The Rector makes the final decision on the appointment of a scientific-pedagogical worker by the Statute of ONMU.
The information system software allows you to enter the following initial data on the university’s scientific-pedagogical staff and calculate the effectiveness of their work according to the KPIs (Table 3).
Figure 2 shows the results of processing assessments of indicators of the seven scientificpedagogical workers of the university based on the use of software for assessing the activity of scientific-pedagogical staff based on KPIs at ONMU.
In Figure 3 you can see the summary KPI, which shows that scientific-pedagogical workers numbered 1, 6 and 4 are the most attractive to the employer in terms of their professional activity, as well as the key performance indicators achieved during the reporting period.
This information system for analyzing key performance indicators objectively evaluates university staff based on the results of their fulfillment of the basic and additional obligations they assumed during the reporting period. The use of an information-analytical system for assessing scientific and pedagogical workers of ONMU based on KPIs will allow for annual monitoring of the performance of university personnel, as well as for management to make objective decisions regarding the results of the competitive selection of employees for vacant positions.
The conducted research was aimed at solving the actual task of using information technologies in university management. To increase the rate of development of the professional level of scientific-pedagogical staff, improve the functioning of ONMU and make effective management decisions, it was proposed to develop an information-analytical system for evaluating the activity of scientific-pedagogical staff based on KPIs. The types of basic and additional obligations that form the technical task of the information-analytical assessment system are determined by the Academic Council and put into effect by the order of the Rector.
The technical task for the information-analytical system for evaluating the activities of scientific-pedagogical staff based on KPIs at ONMU is based on the use of project-oriented management models, the conformity of the content of the assessment with the priority areas of development of the university, the objectivity and reliability of the information obtained from the information-analytical system for evaluating the activity of scientific-pedagogical staff, stimulating scientific-pedagogical staff to improve the results of their professional activities. Based on the proposed technical task, an information-analytical system algorithm was developed, which illustrated the main stages of the system's functioning and established the goal, requirements and output data necessary for the development of an information-analytical system for evaluating the activities of scientific-pedagogical staff based on KPIs. [15] A. Biloshchytskyi, S. Omirbayev, A. Mukhatayev, S. Biloshchytska, S. Toxanov, A.
Faizullin, The concept of the Internet of Things in the development of information and analytical systems based on the method of constructing a scalar assessment of the results of research activities of scientists, PCS 231 (2024) 684-690. doi: 10.1016/j.procs.2023.12.161. [16] Taras Dyda, Nataliia Kunanets, Roman Vaskiv, Daniyar Dauletiya, Sergiy Chernov, Ludmila Chernova, Lubava Chernova, The “Study Easy” Information System, Procedia Computer Science 231 (2024) 678-683. doi: 10.1016/j.procs.2023.12.162. [17] Kamakhya Narain Singh, Jibendu Kumar Mantri, An intelligent recommender system using machine learning association rules and rough set for disease prediction from incomplete symptom set, Decision Analytics Journal 11 (2024) 100468. doi: 10.1016/j.dajour.2024.100468. [18] R. Trach, S. Bushuyev, Analysis of communication network of the construction project participants, Scientific Review Engineering and Environmental Sciences 29 (3) (2020) 388 – 396. doi: 10.22630/PNIKS.2020.29.3.33. [19] A. Voitushenko, S. Bushuyev, Development of project managers’ creative potential: Determination of components and results of research, Advances in Intelligent Systems and Computing 1080 (2020) 283 – 292. doi: 10.1007/978-3-030-33695-0_20. [20] O. Zachko, Methodological basis of safety-oriented management of complex systems development projects, Management of complex systems development 23 (1) (2020) 51-55. [21] O. Kyryllova, V. Shakhov, O. Rossomakha, O. Lozytskyy, S. Zakrevskyi, V. Piterska, Evaluation of the Quality of Research Projects of Universities of Transport Engineering Based on the Portfolio Management Information System, in: 2023 IEEE 18th International Conference on Computer Science and Information Technologies (CSIT), Lviv, Ukraine, 2023, pp. 1-4, doi: 10.1109/CSIT61576.2023.10324064. [22] V. Piterska, V. Samoilovska, V. Adakhovskyi, Assessment of Port Concession Projects Quality Based on the Information and Analytical Risk Management System, in: Proceedings of the 4th International Workshop IT Project Management (ITPM 2023), CEUR Workshop Proceedings, 2023, vol. 3453, pp. 71-81. [23] V. Samoilovska, O. Kyryllova, V. Piterska, Model for Evaluating the Efficiency of Seaports Development Projects Based on the Quality 4.0 Information and Analytical System, in: Proceedings of the 4th International Workshop IT Project Management (ITPM 2023), CEUR Workshop Proceedings, 2023, vol. 3453, pp. 1-12. [24] A. Bondar, S. Bushuyev, S. Onyshchenko, H. Tanaka, Entropy paradigm of projectoriented organizations management, in: Proceedings of the 1st International Workshop ITPM 2020, 2020, CEUR Workshop Proceedings, vol. 2565, 2020, pp. 233-243. [25] S. Bushuyev, S. Onyshchenko, N. Bushuyeva and A. Bondar, "Modelling projects portfolio structure dynamics of the organization development with a resistance of information entropy," 2021 IEEE 16th International Conference on CSIT, 2021, pp. 293-298, doi: 10.1109/CSIT52700.2021.9648713. [26] O. Sherstyuk, T. Olekh, and K. Kolesnikova, The research on role differentiation as a method of forming the project team, EEJET 2 3(80) (2016) 63–68. [27] V. Piterska, A. Shakhov, O. Lohinov and L. Lohinova, The Method of Human Resources Management of Educational Projects of Institution of Higher Education, in: 2020 IEEE 15th International Conference on Computer Sciences and Information Technologies (CSIT), 2020, pp. 123-126, doi: 10.1109/CSIT49958.2020.9321912. [28] V. Piterska, A. Shakhov, O. Lohinov and L. Lohinova, The Method of Transfer of Research Project Results of Institution of Higher Education, in: 2019 IEEE 14th International Conference on Computer Sciences and Information Technologies (CSIT), 2019, pp. 77-80, doi: 10.1109/STC-CSIT.2019.8929887. [29] V. Piterska, D. Lohinov and L. Lohinova, Risk Management Mechanisms in Higher Education Institutions Based on the Information Support of Innovative Projects, in: 2022 IEEE 17th International Conference on Computer Sciences and Information Technologies (CSIT), 2022, pp. 410-413, doi: 10.1109/CSIT56902.2022.10000551. [30] O. Kyryllova, V. Piterska, O. Sagaydak, O. Lohinov, L. Lohinova, Conceptual model of educational project management related to maritime transport in higher education institutions, in: Proceedings of the 3rd International Workshop IT Project Management (ITPM 2022), CEUR Workshop Proceedings, 2022, 3295, pp. 138–148. [31] S. Rudenko, A. Shakhov, V. Piterska, L. Chernova and O. Sherstiuk, Application of balanced scorecard for managing university development projects, in: 2021 IEEE 16th International Conference on Computer Sciences and Information Technologies (CSIT), 2021, pp. 311-314, doi: 10.1109/CSIT52700.2021.9648580. [32] O. Shumylo, V. Yarovenko, M. Malaksiano, O. Melnyk, Comprehensive Assessment of Hull Geometry Influence of a Modernized Ship on Maneuvering Performance and Propulsion System Parameters, Pomorstvo 37(2) (2023) 314–325. doi: 10.31217/p.37.2.13. [33] V. Romanuke A. Romanov, M. Malaksiano, A Crossover Operators in a Genetic Algorithm for Maritime Cargo Delivery Optimization, Journal of ETA Maritime Scienc 10(4) (2022) 223–236. doi: 10.4274/jems.2022.80958. [34] V. Romanuke, A. Romanov, M. Malaksiano, Pseudorandom number generator influence on the genetic algorithm performance to minimize maritime cargo delivery route length, Pomorstvo 36(2) (2022) 249–262. doi: 10.31217/p.36.2.9. [35] A. Tryhuba, O. Bashynsky, Coordination of dairy workshops projects on the community territory and their project environment, in: 2019 IEEE 14th International Conference on CSIT, 2019, pp. 51-54, doi: 10.1109/STC-CSIT.2019.8929816. [36] A. Tryhuba and O. Bashynsky, Coordination of dairy workshops projects on the community territory and their project environment, in: 2019 IEEE 14th International Conference on CSIT, 2019, pp. 51-54, doi: 10.1109/STC-CSIT.2019.8929816. [37] D. Kobylkin, O. Zachko, R. Ratushny, A. Ivanusa, C. Wolff, Models of content management of infrastructure projects mono-templates under the influence of project changes, CEUR Workshop Proceedings, vol. 2851, 2021, pp. 106-115. [38] B. S. Akhmetov, V. Lakhno, B. B. Akhmetov, A. Zhilkishbayev, N. Izbasova, O.
Kryvoruchko, A. Desiatko, Application of a Genetic Algorithm for the Selection of the Optimal Composition of Protection Tools of the Information and Educational System of the University, PCS 215 (2022) 598-607. doi: 10.1016/j.procs.2022.12.062. [39] S. Guzmán-Castillo, F. Körner, J. I. Pantoja-García, L. Nieto-Ramos, Y. Gómez-Charris, A. Castro-Sarmiento, A. R. Romero-Conrado, Implementation of a Predictive Information System for University Dropout Prevention, PCS 198 (2022) 566-571. [40] P. Udupa, Application of artificial intelligence for university information system, EAAI 114 (2022). doi: 10.1016/j.engappai.2022.105038.