Intelligent Systems for Assessment and Development of Personnel: Mentoring Study Nestor Shpak1, Kateryna Doroshkevych1, Zoriana Dvulit1, Roman Dzvonyk1 and Andriy Bosak1 1 Lviv Polytechnic National University, St. Bandery str., 12, Lviv, Ukraine, 79013 Abstract The article investigates the intellectual systems intended for the evaluation and development of personnel of the enterprise (intelligent HR systems) and emphasizes the insufficient level of their development in the conditions of mentoring implementation. To strengthen the motivational impact on employees and increase the level of enterprise management in general, the article proposes to develop intelligent systems in the direction of mentoring assessment, for which an appropriate evaluation procedure has been formed. As part of the proposed procedure, a scorecard is recommended, which provides for assessing the level of mentoring processes, identifying the effectiveness of mentoring. To identify the difference in the level of involvement in the mentoring processes of various structural units of the enterprise (departments), the article provides posterior comparisons according to the Tukey HSD. Practical approbation of the order took place in the Lviv IT cluster, where there are minor differences in the levels of mentoring at enterprises. Keywords 1 Intelligent systems, personnel, development, assessment, mentoring, Tukey HSD 1. Introduction As you know, intelligent systems provide a standardized methodological approach to solving important and rather complex problems and allow you to get consistent and reliable results over time. [1]. They are widely used in enterprise management for: planning and scheduling of product development projects, downloading of production capacities of manufacturing enterprise, new product development and selecting a new product portfolio, evaluation of human resource strategies, recruitment and promotion, manufacturing design (based on the set of very specialized services that could be arranged to provide new creative and sustainable processes, etc.), the implementation of the lean maintenance concept, which allows to increase the operational efficiency of the company's technical infrastructure, etc. [2-7]. Characteristic of them is the use of artificial intelligence that can simulate intelligent functions. As stated in [9], are large-scale application software packages that support business processes and the flow of information as well as reporting and data analytics in firms as organizations. When managing personnel, they provide: management of the organizational structure and staffing; calculation of wages; personnel accounting; time tracking; planning of personnel costs; career planning and tracking the promotion of personnel in the structure; work with personnel reserve; recruitment of personnel for vacancies; staff training; attestation systems; competency management. Thus, a modern intelligent HR system is a complex of technologies that automate and facilitate work with personnel, ranging from everyday data accounting, ending with strategic decisions on the COLINS-2022: 6th International Conference on Computational Linguistics and Intelligent Systems, May 12–13, 2022, Gliwice, Poland EMAIL: nestor.o.shpak@lpnu.ua (N. Shpak); kateryna.o.doroshkevych@lpnu.ua (K. Doroshkevych); zoriana.p.dvulit@lpnu.ua (Z. Dvulit); roman.y.dzvonyk@lpnu.ua (R. Dzvonyk); andrii.o.bosak@lpnu.ua (A. Bosak). ORCID: 0000-0003-0620-2458 (N. Shpak); 0000-0003-3966-224X (K. Doroshkevych); 0000-0002-2157-1422 (Z. Dvulit); 0000-0002- 1702-0775 (R. Dzvonyk); 0000-0002-2944-2166 (A. Bosak). ©️ 2022 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). CEUR Workshop Proceedings (CEUR-WS.org) development of the enterprise. At the same time, mentoring processes remain outside the functionality of the intelligent system, although they have a significant impact on the development of knowledge in the enterprise. Let us consider it more carefully. Mentoring as a method of personalized learning has a significant motivational effect and requires an appropriate level of enterprise costs. The economic feasibility of activating and applying mentoring at the enterprise as a type of training and personnel development can be identified in the evaluation process. It, among other things, consists in measuring the effects, effectiveness, long-term consequences of processes and phenomena, etc. The results obtained during the evaluation process serve to achieve the goals of developing mentoring programs [11]. So, in addition to economic efficiency, in the process of evaluation, it is possible to identify the effectiveness of mentoring, the level of involvement of employees (the intensity of mentoring influence), its significance, acceptability in view of specific conditions for the implementation of production and economic activities, etc. Thus, we consider the evaluation of mentoring important for the implementation of effective management of the enterprise in the conditions of innovative development. Management decisions on activation, formation of mentoring program, its development, termination of implementation at the enterprise, etc. depend on its results. That is why the evaluation process requires systemicity and structure, as well as proper information support. Consider the processes of assessing mentoring and other methods of training, mastering new professional skills (approaches, introducing new technologies of work), business qualities of employees, the level of HR management, etc. On their basis, we will form an architecture that could complement the functionality of intelligent systems designed for the evaluation and development of enterprise personnel (intelligent HR systems). 2. Related works In economic literature, there is no single toolbox designed to evaluate the results of mentoring. For this purpose, observations, checklists, questionnaires, oral surveys, expert evaluation, analysis of final works, Color test of relations, Psychometric scale of Likert, Utrecht scale of involvement in the work of V. Shawfeeli, etc. are used [12-16]. Most of these methods are implemented individually, or are part of the methods that involve the use of the system of evaluation indicators. For example, the evaluation stage of mentoring technology according to [17] involves testing, case tasks and questionnaires to provide feedback that can be made using a competency diagnostic checklist (example of which is given by the author). This will allow identify the skills of employees received in the process of mentoring and its effectiveness on the part of the mentor (employees). According to the recommendations provided by Daniali S., Khortabi F. M., Mohammadbeiki Y., Ilyushnikov K. K., Lobova S. V. [18, 19] to assess the level of satisfaction of the head of personnel training should apply the questionnaire of O. Vetluzhskyh [20]. At the same time, this indicator (the level of satisfaction of the head with personnel training) is included in the system of indicators and criteria for assessing the effectiveness of personnel training for railway enterprises. It also includes the following indicators: implementation of the training plan; staff training costs; loyalty index; use of knowledge gained in professional activities; increase in productivity; technological violations by fault of employees, etc. To calculate other indicators, it is provided to calculate using formulas and apply expert assessments. HR management assessment indicators contain a number of evaluation indicators that are summarized in three directions: point to the level of organizational management, characterize technological and educational management [21]. To evaluate each of the areas, the following indicators are recommended: high-quality staffing; level of staff skillset; staff stability factor; emotional intelligence; the level of compliance of personnel goals with business goals; degree of satisfaction with work; level of social tension and conflict; the level of transformation of roles in the team; employment rate of all roles (according to the Belbin model), etc. The methodology for calculating indicators provides for expert assessments, mathematical calculation of quantitative indicators and questionnaires (for example, in the assessment of emotional intelligence). In scientific work [22] to evaluate the effectiveness of the program development of personnel of the organization, it is recommended to apply an approach according to which the economic effect will be determined as the difference between changes in production and training costs. The author recommends various approaches to assessing the effectiveness of development for the following cases: training is necessary for the development of a new profession or position in order to further combine positions; training of employees, which is prescribed by law and controlled by the authorities (in this case, the economic effect is proposed to be defined as an economic assessment of the consequences of not conducting this training); assessment of the effect of training non-production personnel; effectiveness of the personnel development system. Thus, the effectiveness of personnel development systems can be measured, both in absolute monetary terms and in the form of relative indicators. To assess the effectiveness of the author's methodology for building an individual trajectory of self-study, Borisov I.V. proposes to use a six-component model of involvement, which provides for appropriate engagement indicators and a methodology for their calculation, involving the use of diagnostic procedures using questionnaires and questionnaires, criteria for their calculation. Further, to obtain reliable and representative results of the evaluation, the author carried out a statistical analysis of the obtained results, which provides differential analysis and a posterior comparisons using the Tukey criterion [12]. Rumi Agarwal,·Laura Heron, Mitra Naseh, Shanna L. Burke used online platforms (Research Electronic Data Capture (RedCap)), in our research to gather information, where we were interviewed in the process of training. Then there was a statistical analysis of the obtained data by conducting paired sample t-tests, followed by correction of Bonferroni to paired samples of t-tests. Statistical Package for Social Sciences V.20 (SPSS; IBM Corp. 2017) was used by the authors to analyse quantitative data. [23]. Paired t-tests are also used in evaluating the effectiveness of the group mentoring model (a learning model that provides group collaboration using each other's teaching methods, demonstrating and modelling recently acquired knowledge and skills). In this case, researchers used observation, surveys, testing for information support [24]. In order to collect information about the results of mentoring, online platforms were used by the Fremantle School of Medicine at the University of Notre Dame (Australia). As part of the study, university students used their electronic portfolios (supported by Blackboard) to provide feedback in the process of their studies [25]. Reviews of mentoring programs (scores) collected from the electronic portfolio of students are further statistically investigated. It is also possible to summarize the results of employee questionnaires using their personal corporate e-mail boxes, as it is done in [26]. A thorough study of the mentoring techniques found in the economic literature on the problem of mentoring undergraduate students [27]. The author studied 80 literary sources in 2013-2020, which made it possible to summarize the theoretical and conceptual field, data collection methods, obtained results, etc. In addition to the above methods of collecting data on mentoring, the author noted the use of individual and focus groups, record logs, written minds of students, ranking (on the persuasion scale, Likert), mental health testing, etc. In [28] focus groups are combined with the survey and the use of study diaries. 3. Methods From the study we conducted, it can be argued that the most common in the practice of determining the results of personalized learning is to survey mentoring participants using a standardized questionnaire (questionnaire) with answers posted on the Likert scale. This approach is used, for the most part, to assess the relevance of the mentoring program and its impact on the professional activity of the mentee. This approach allows us to identify the impact of mentoring programs on the activities of relevant groups of employees (doctors, students, etc.). In particular, in [29] it is established that in addition to the described positive impact, mentoring also serves as psychosocial support, creating free access between the mentee and the mentor reflected in the excellent relationship between them. In the conditions of the proper level of implementation of the enterprise's information systems, information support for the evaluation of mentoring is carried out using online services, electronic boxes of mentees and mentors, etc., which creates conditions for automated processing of survey data (questionnaires). This is done by collecting expert opinions on various aspects of mentoring activities that are not measurable, but are used in calculating indicators. In the practice of enterprise management, it is common to calculate indicators by which the effectiveness of mentoring (coaching, mentorship) is revealed. In order to obtain accurate and reliable results of the assessment, the calculated indicators and results of the questionnaire are subject to further processing using statistical (differential) analysis. This will ensure the adoption of rational management decisions based on the results of the implemented study. Approaches to the evaluation of mentoring in enterprises are summarized in Fig. 1. Approaches to mentoring assessment Rating indicators system Questionnaires, surveys, focus groups, etc. Statistical and Expert Calculation of differential assessments indicators analysis Figure 1: Approaches to assessing mentoring in enterprises When evaluating mentoring activities in an organization, the following should be taken into account. First, the business qualities of the staff are subject to evaluation. According to the recommendations provided in the [30], the system of assessing the business qualities of personnel should be carried out in accordance with the directions of the organization and their most significant property, include standards, criteria of effectiveness, as well as the established procedure for calculating valuation points, take measures based on the results of the assessment. Based on the competent approach, the main principles of mentoring assessment are: objectivity; reliability; predictability; complexity; accessibility and openness; systematic; effectiveness and efficiency. On the basis of generalization of approaches to the evaluation of mentoring (Fig. 1) in order to ensure the effectiveness of the assessment, the availability of results for use and compliance with the principles of mentoring, we propose to implement the following procedure of evaluation: determining the purpose and objectives of mentoring evaluation; information support of evaluation processes; selection of methods intended for assessing mentoring in the enterprise; evaluation of mentoring by implementing the selected method (set of methods); generalization of results and management decisions (Fig. 2). Determining the purpose and objectives of mentoring assessment Information support of mentoring evaluation processes Selection of methods intended for assessing mentoring at the enterprise Evaluation of mentoring by implementing the selected method (set of methods) Generalization of results and management decisions Figure 2: Recommended procedure for assessing mentoring activities at the enterprise We will reveal each of the stages of the recommended order. 4. Results and Discussion The purpose of the mentoring evaluation processes we discussed above, it can be realized by performing a number of evaluation tasks. We prioritize two main tasks: determining the level of mentoring processes at the enterprise and identifying the effectiveness of mentoring activities. The first task is to identify how implemented the mentoring process at the enterprise, the employees involved in the mentoring process, the plan of personalized training, the mastered budget of mentoring activities, etc. The second task is to evaluate the results of mentoring activities, covering the achievement of employees' goals, raising the level of knowledge and their satisfaction with work, increasing the innovative level of the enterprise, etc. Information support consists in collecting, processing information that is necessary for the implementation of assessment tasks. As you know, the effectiveness of this stage is determined by the use of appropriate information technologies. Since not all aspects of mentoring activities at the enterprise are subject to quantitative measurement, questionnaires, surveys, examinations, etc. can be used to assess mentoring. In this case, information support can be developed in the direction of using online questionnaires, personal accounts of employees, online testing, etc. This will speed up the processes of collecting, classifying, storing information, etc. Among the methods intended for assessing mentoring at the enterprise, we recommend using a system of evaluation indicators that will ensure quantitative assessment, reliability (objectivity), identify the level of achievement of goals and reserves of mentoring activities and develop directions for their application, etc. It should be noted that the optimal number of indicators is 5-25 pcs., they should all be quantitatively measured (in particular, as a result of expert assessments) and criterion (the obtained value of the indicator should indicate the level of achievement of the corresponding mentoring goal). We recommend a system of indicators covering 2 areas of evaluation: the level of mentoring processes in the enterprise, the effectiveness of the company's mentoring (Table 1). Table 1 The system of indicators intended for the evaluation of mentoring activities in enterprises № Indicators Essence Calculation Criteria values 1 2 3 4 5 Assessment of the level of mentoring processes in the enterprise 1 Mentoring Plan The ratio of the actual 𝑅𝑒𝑎 [0; 0,55] – non-fulfilment of 𝑀𝑝𝑒𝑙 = Execution Level number of employees 𝑅𝑒𝑎𝑝 the plan; [0,56; 0,75] - (𝑀𝑝𝑒𝑙 ) involved in the mentoring partial implementation of process (𝑅𝑒𝑎 ) to planned the plan; [0,76; 1] - execute (𝑅𝑒𝑝 ) the plan 2 The level of Indicates the ratio of 𝐶𝑡𝑒𝑓 [0; 0,55] – minimum cost 𝐿𝑡𝑒𝑒 = expenses for actual costs made to 𝐶𝑡𝑒𝑝 level; [0,56; 0,75] – average mentoring mentoring and other types cost level;[0,76; 1] – activities and of training in the optimal cost level other types of enterprise (𝐶𝑡𝑒𝑓 ) to training of planned, which should be employees 5% of the remuneration (𝐿𝑡𝑒𝑒 ) fund (𝐶𝑡𝑒𝑝 ) 3 Employee It is calculated similarly to 𝐼𝑒𝑙 =𝑆𝑠𝑚𝑒 − [0; 0,55] – mentoring loyalty index to the Employee Loyalty 𝑆𝑐𝑚𝑒 program should be mentoring (𝐼𝑒𝑙 ) Index (eNPS), which replaced; [0,56; 0,75] – implies a difference mentoring program should between the share of be adjusted; [0,76; 1] – supporters (𝑆𝑠𝑚𝑒 ) and mentoring program meets mentoring critics among the goals and objectives of employees (𝑆𝑐𝑚𝑒 ) the enterprise 1 2 3 4 5 4 Managers' It is calculated similarly to 𝐼𝑚𝑙 =𝑆𝑠𝑚𝑚 − [0; 0,55] – managers are loyalty index to the Employee Loyalty 𝑆𝑐𝑚𝑚 completely dissatisfied with employee Index (eNPS), which the program; [0,56; 0,75] – mentoring (𝐼𝑚𝑙 ) provides for the difference managers are partially between the share of satisfied with the supporters (𝑆𝑠𝑚𝑚 ) and mentoring program; mentoring critics (𝑆𝑐𝑚𝑚 ) [0,76; 1] – managers are among managers whose loyal to the mentoring staff has been trained program Evaluation of the effectiveness of mentoring in the enterprise 5 The level of The ratio of the number of 𝑁𝑠𝑝 [0; 0,55] – low stability of 𝐿𝑠𝑐𝑝 = stability of the employees who work at 𝑇𝑛𝑒 the company's personnel; company's the enterprise for more [0,56; 0,75] – average personnel than a year (𝑁𝑠𝑝 ) to the stability of the company's (𝐿𝑠𝑐𝑝 ) total number of employees personnel; [0,76; 1] – of the enterprise (𝑇𝑛𝑒 ) stable staff 6 Level of Ratio of the number of 𝑁 𝑒𝑖𝑞 [0; 0,55] – low level of 𝐿𝑝𝑑𝑒 = professional employees who improved 𝑇𝑛𝑒 professional development development of their qualifications in the of employees; [0,56; 0,75] – employees reporting period (𝑁𝑒𝑖𝑞 ) to average level of (𝐿𝑝𝑑𝑒 ) the total number of professional development employees of the of employees; [0,76; 1] – enterprise (𝑇𝑛𝑒 ) high level of professional development of employees 7 The level of The ratio of the number of 𝑁𝑒𝑖𝑖 [0; 0,55] – low level; [0,56; 𝐿𝑒𝑖𝑖 = involvement of employees involved in 𝑇𝑛𝑒 0,75] – average level; [0,76; personnel in innovation activities (𝑁𝑒𝑖𝑖 ) 1] – high level of innovation to the total number of involvement of employees (𝐿𝑒𝑖𝑖 ) employees (𝑇𝑛𝑒 ) in innovation activities 8 The level of The ratio of the number of 𝑁𝑗𝑎𝑞 [0; 0,55] – low level; [0,56; 𝐿𝑞𝑠𝑒 = quantitative employees who took jobs 𝑇𝑛𝑒 0,75] – average level; [0,76; staffing of the of appropriate 1] – high level of staffing of enterprise qualification (𝑁𝑗𝑎𝑞 ) to the the state of the enterprise states (𝐿𝑞𝑠𝑒 ) total number of employees of the enterprise (𝑇𝑛𝑒 ) 9 Average goal Indicates the level of Expert 0 – employee's goals do not match rank compliance of personnel evaluation meet the goals of the (𝐴𝑔𝑚𝑟 ) goals with the goals of the according to enterprise; 0,5 – partial enterprise the specified compliance with the goals criteria of the employee and the enterprise; 1 – employee's goals fully meet the goals of the enterprise 10 The level of Indicates the ratio of 𝑅𝑝𝑓 [0; 0,55] – the level of 𝐿𝑖𝑝 = increase in actual productivity (𝑅𝑝𝑓 ) 𝑅𝑝𝑝 increase in production is productivity to the planned one, which minimal; [0,56; 0,75] – (production) at should be 5% more than average level; [0,76; 1] – the enterprise the productivity in the the level of increase in (𝐿𝑖𝑝 ) base period (𝑅𝑝𝑝 ) production is optimal The information base for calculating indicators is the data of analytical and management accounting of enterprises, as well as expert assessments. On their basis, indicators should be calculated, as well as generalizations should be carried out (for example, determining the average level as provided for in calculating the average rank of compliance with goals, etc.). In order to summarize the results of the assessment, we propose to identify the integral level of indicators (𝐼𝑚𝑎 ), taking into account their weight. This is ensured in the process of using the factorial method by the formula: 𝐼𝑚𝑎 = 𝑀𝑝𝑒𝑙 × 𝑘𝑝𝑒𝑙 + 𝐿𝑡𝑒𝑒 × 𝑘𝑡𝑒𝑒 + 𝐼𝑒𝑙 × 𝑘𝑒𝑙 + 𝐼𝑚𝑙 × 𝑘𝑚𝑙 + 𝐿𝑠𝑐𝑝 × 𝑘𝑠𝑐𝑝 + +𝐿𝑝𝑑𝑒 × 𝑘𝑝𝑑𝑒 + 𝐿𝑒𝑖𝑖 × 𝑘𝑒𝑖𝑖 + 𝐿𝑞𝑠𝑒 × 𝑘𝑞𝑠𝑒 + 𝐴𝑔𝑚𝑟 × 𝑘𝑔𝑚𝑟 + 𝐿𝑖𝑝 × 𝑘𝑖𝑝 , (1) where 𝑘𝑖 – weight of indicators intended for the evaluation of mentoring activities in enterprises. ∑10 𝑖=1 𝑘𝑖 = 1. According to the recommendations provided in the economic literature on identifying the level of involvement of individuals in the learning process, the results obtained should be checked for statistical effects that can be evaluated using two-factual dispersion analysis of all components of involvement in mentoring processes [12]. To identify the difference in the level of involvement of various structural units of the enterprise (departments) in the mentoring processes. In order to identify differences in the statistically significant studied criteria of the groups (departments) and posterior comparisons should be made according to the Tukey criterion [31-35]. As a result, we will find out which departments (enterprises) differ in the level of implementation of individual mentoring components that characterize the recommended indicators (Table 1). 4. Experimental We will evaluate mentoring activities at Lviv IT cluster enterprises. Lviv IT Cluster is a community of IT companies, government and education that have taken the responsibility to work on the development of the industry and the region through education, industry promo (city, region and companies) and infrastructure [36]. Cluster participants are companies of different sizes and with different needs, most of them have experience in mentoring activities that should be evaluated. Thus, the presence of different enterprises in the cluster allows you to evaluate individually by enterprises and identify the level of involvement of employees in mentoring activities in the cluster. To do this, we calculate the indicators intended for assessing mentoring at the enterprises of the cluster (Table 2). Table 2 Values of indicators intended for evaluation of mentoring activities at Lviv IT cluster enterprises № Indicators Value in enterprises #1 #2 #3 #4 Assessment of the level of mentoring processes in the enterprise 1 Mentoring Plan Execution Level (𝑀𝑝𝑒𝑙 ) 0,82 0,84 0,76 0,8 2 The level of expenses for mentoring activities and other types of 0,9 0,86 0,92 0,95 training of employees (𝐿𝑡𝑒𝑒 ) 3 Employee loyalty index to mentoring (𝐼𝑒𝑙 ) 0,56 0,82 0,73 0,58 4 Managers' loyalty index to employee mentoring (𝐼𝑚𝑙 ) 0,72 0,68 0,59 0,8 Evaluation of the effectiveness of mentoring in the enterprise 5 The level of stability of the company's personnel (𝐿𝑠𝑐𝑝 ) 0,53 0,75 0,48 0,6 6 Level of professional development of employees (𝐿𝑝𝑑𝑒 ) 0,86 0,72 0,76 0,84 7 The level of involvement of personnel in innovation ( 𝐿𝑒𝑖𝑖 ) 0,76 0,78 0,8 0,78 8 The level of quantitative staffing of the enterprise states (𝐿𝑞𝑠𝑒 ) 0,86 0,9 0,79 0,82 9 Average goal match rank (𝐴𝑔𝑚𝑟 ) 1 1 0,5 1 10 The level of increase in productivity (production) at the enterprise 0,63 0,5 0,76 0,89 (𝐿𝑖𝑝 ) Lviv IT Cluster mission: "In the future, we see Lviv as a world-class high technology center. And our mission is to contribute to this as much as possible." The cluster includes companies, educational institutions, local authorities and BPO participants. According to this we will carry out an expert assessment of the weight of each of the indicators included in the recommended system (Table 1). This will reveal the level of mentoring activity at each of the enterprises of the Lviv IT cluster. The results are summarized in Table. 3. Table 3 Weight of indicators intended for evaluation of mentoring activities at Lviv IT cluster enterprises № Indicators Weight Assessment of the level of mentoring processes in the enterprise 1 Mentoring Plan Execution Level (𝑀𝑝𝑒𝑙 ) 0,13 2 The level of expenses for mentoring activities and other types of training of 0,03 employees (𝐿𝑡𝑒𝑒 ) 3 Employee loyalty index to mentoring (𝐼𝑒𝑙 ) 0,16 4 Managers' loyalty index to employee mentoring (𝐼𝑚𝑙 ) 0,05 Evaluation of the effectiveness of mentoring in the enterprise 5 The level of stability of the company's personnel (𝐿𝑠𝑐𝑝 ) 0,13 6 Level of professional development of employees (𝐿𝑝𝑑𝑒 ) 0,03 7 The level of involvement of personnel in innovation (𝐿𝑒𝑖𝑖 ) 0,16 8 The level of quantitative staffing of the enterprise states (𝐿𝑞𝑠𝑒 ) 0,05 9 Average goal match rank (𝐴𝑔𝑚𝑟 ) 0,1 10 The level of increase in productivity (production) at the enterprise (𝐿𝑖𝑝 ) 0,12 We base the obtained values of meaning of indicators (Table 2) and weight for enterprises of Lviv IT cluster (Table 3) in the equation (1). As a result, the level of mentoring activity (𝐼𝑚𝑎 ) of each of the enterprises was obtained: for the enterprise 1 𝐼𝑚𝑎 =0,74; for the enterprise 2 𝐼𝑚𝑎 =0,76; for the enterprise 3 𝐼𝑚𝑎 =0,67; for the enterprise 4 𝐼𝑚𝑎 =0,79. Next, we will compare the levels of mentoring of enterprises in the Lviv IT cluster using the online service (https://astatsa.com/OneWay_Anova_with_TukeyHSD/), which conducts posterior comparisons according to the Tukey criterion. The results are summarized in Table. 4. Table 4 Posterior comparisons of the level of mentoring activity at the enterprises of Lviv IT cluster according to the Tukey criterion Tukey HSD Tukey HSD Tukey HSD Treatments pair Q statistic p-value inference Enterprise 1 and insignificant 0.1636 0.8999947 Enterprise 2 Enterprise 1 and insignificant 0.4658 0.8999947 Enterprise 3 Enterprise 1 and insignificant 0.4127 0.8999947 Enterprise 4 Enterprise 2 and insignificant 0.6294 0.8999947 Enterprise 3 Enterprise 2 and insignificant 0.2490 0.8999947 Enterprise 4 Enterprise 3 and insignificant 0.8785 0.8999947 Enterprise 4 As a result of the calculations, we can affirm insignificant differences in the levels of mentoring for enterprises of the Lviv IT Cluster and a sufficient level of mentoring activities of the cluster. This is also evidenced by the data obtained as a result of the aggregation of indicators and the determination of the integral level of mentoring activities. 5. Conclusions The article substantiated the importance of assessing mentoring as a method of personalized learning, which has a significant motivational impact on the employees of the enterprise and the effectiveness of their activities. To ensure this, approaches to the evaluation of mentoring at enterprises have been studied and summarized, as a result of which the procedure for evaluating mentoring activities and the scorecard are recommended. The recommended procedure contains the stages of mentoring assessment, and the indicators are grouped into two areas of evaluation: assessment of the level of mentoring processes in the enterprise (level of implementation of the mentoring plan, the level of costs for mentoring activities and other types of training of employees, the index of loyalty of employees to mentoring, the index of loyalty of managers to mentoring employees), determining the effectiveness of mentoring the enterprise (level of stability of the company's personnel, the level of professional development of employees, the level of involvement of personnel in innovation, the level of quantitative complexity of the states of the enterprise, the average rank of compliance with goals, the level of increase in productivity (production) in the enterprise). The recommended procedure provides for summarizing indicators and calculating the integral level of mentoring activities and identifying differences in the level of involvement in the mentoring processes of various structural units of the enterprise (departments), for which a posterior comparisons are provided according to the Tukey criterion. Practical approbation of the order took place on the basis of the Lviv IT Cluster, where there are minor differences in the levels of mentoring at enterprises. Further research should explore the possibility of combining the developed order of mentoring evaluation with intelligent HR systems in the enterprise (DeloPro, SAP R/3, Baan, Oracle Applications, etc.). 6. References [1] T. A. Byrd, R. D. Hauser, Expert systems in production and operations management: research directions in assessing overall impact, Int. J. Prod. Res., Volume 29, (1991) 2471-2482. [2] M. Relich, W. Muszyński, The Use of Intelligent Systems for Planning and Scheduling of Product Development Projects, Procedia Computer Science, Volume 35, (2014) 1586-1595. [3] M. Relich, K. Bzdyra, Estimating new product success with the use of intelligent systems, Foundations of Management, Volume 6, 2 (2014) 7-20. [4] Y. Wang, L. Li, L. Yang, Intelligent Human Resource Planning System in a Large Petrochemical Enterprise, IEEE Intelligent Systems, Volume 28, 4 (2013) 102-106. DOI: 10.1109/MIS.2013.112. [5] J. R. Silva, New Trends in Manufacturing: Converging to Service and Intelligent Systems, IFAC Proceedings, Volume 47, 3 (2014) 2628-2633, URL: https://doi.org/10.3182/20140824-6-ZA- 1003.02823. [6] K. Antosz, L. Pasko, A. Gola, The Use of Intelligent Systems to Support the Decision-Making Process in Lean Maintenance Management, IFAC-Papers, Volume 52, 10 (2019) 148-153. URL: https://doi.org/10.1016/j.ifacol.2019.10.037. [7] N. Shpak, M. Odrekhivskyi, K. Doroshkevych, W. Sroka, Simulation of Innovative Systems under Industry 4.0 Conditions, Social Science 8 (7) (2019) 202. DOI: 10.3390/socsci8070202. [8] N. Shpak, N. Podolchak, V. Karkovska, W. Sroka. The Influence of Age Factors on the Reform of the Public Service of Ukraine, Central European Journal of Public Policy, 13(2), (2019) 40– 52. DOI: 10.2478/cejpp-2019-0006. [9] A. Mouzoune, Contribution to Enterprise Intelligent Systems Architecture: Assumptions, Expectations and a Proposal, Intelligent Information Management, Vollume 4, 6 (2012) 1-8. DOI:10.4236/iim.2012.46035. [10] N. Shpak, K. Doroshkevych, Y. Shpak, I. Salata, M. Sharko, Strategy and Tactics of International Digitalization and Intellectualization of Economic Relations, CEUR Workshop Proceedings: Proceedings of 5th International Conference on Computational Linguistics and Intelligent Systems (COLINS-2021), Volume 2870 (2021) 1477-1487. URL: http://ceur- ws.org/Vol-2870/paper109.pdf. [11] A. J. Hobson, B. Maxwell, K. Káplár-Kodácsy, The Role of the Mentoring, Programme Coordinator: A Rapid Evidence Review Education & Training Foundation, University of Brighton and Sheffield Hallam University, (05) 2021. URL: https://www.et- foundation.co.uk/wp- content/uploads/2021/05/ETF_MCoord_Report_UoB_SHU_29_April_2021_Final.pdf. [12] D. I. Martsenkovskyi, Vikovyi polimorfizm posttravmatychnykh stresovykh rozladiv ta depresii u ditei vnutrishno peremishchenykh osib z okupovanykh terytorii [Age polymorphism of post- traumatic stress disorders and depression in children of internally displaced persons from the occupied territories], Arkhiv psykhiatrii [Archives of Psychiatry], Volume 24, 1 (2018) 15-20. URL: http://nbuv.gov.ua/UJRN/apsuh_2018_24_1_4. [13] J. Batut, M. Kvaskoff, M. C. Morris, When mentoring matters: a French mentoring program for women in science, Nature Biotechnology, 39 (2021) 776–779. URL: https://www.nature.com/articles/s41587-021-00951-2. [14] A. G. Opsahl, C. Townsend, Mentoring to engage diverse undergraduate nursing students in honors research, Nurs Forum, 56:19 (2021). DOI: https://doi.org/10.1111/nuf.12503. [15] D. Games, The Effectiveness of Mentoring and Coaching in Teaching Business Planning: The Case of Universitas Andalas, Advances in Social Science, Education and Humanities Research, Proceedings of the 3rd International Conference on Educational Development and Quality Assurance (ICED-QA 2020), 506 (2020) 1-5. [16] S. Resnjanskij, J. Ruhose, S. Wiederhold, L. Woessmann, Mentoring Improves the Labor-Market Prospects of Disadvantaged Adolescents, CESifo Forum, Volume 22, 4 (2021) 38-73. URL: https://www.cesifo.org/DocDL/CESifo-forum-2021-4-resnjanskij-etal-Mentoring-Improves-the- Labor-Market-Prospects-of-Disadvantaged-Adolescents.pdf. [17] G. I. Egorova, Tehnologicheskij diskurs povyshenija urovnja professionalnyh kompetencij pedagoga [Technological discourse of increasing the level of professional competencies of a teacher], Humanities researches of the Central Russia, 1 (18) (2021) 45-63. URL: https://cyberleninka.ru/article/n/tehnologicheskiy-diskurs-povysheniya-urovnya-professionalnyh- kompetentsiy-pedagoga/viewer. [18] S. Daniali, F. M. Khortabi, Y. Mohammadbeiki, Innovative approach for evaluating training effectiveness, Proceedings of the 32nd International Business Information Management Association Conference, IBIMA 2018 – Vision 2020: Sustainable Economic Development and Application of Innovation Management from Regional expansion to Global Growth, 32 (2018) 818-830. [19] K. K. Ilyushnikov, S. V. Lobova, Evaluation model for effectiveness of corporate training based on key performance indicators and detailed metrics, Espacios, Volume 40, 1 (2019), 29-37. URL:http://www.revistaespacios.com/ a19v40n01/a19v40n01p29.pdf [20] E. N. Vetluzhskih, Obuchaem – ocenivaem. Modeli kriterialnoj ocenki rezultatov obuchenija [Models for Criteria-Based Assessment of Learning Outcomes], Spravochnik po upravleniju personalom [Human Resources Handbook], 2 (2005) 10-18. [21] T. L. Bezrukova et al, Optimization model of motivational process as a basis of management decisions, IOP Conf. Ser.: Mater. Sci. Eng, 483 012025 (2019), DOI:10.1088/1757- 899X/483/1/012025. [22] E. A. Belalova, Metody analiza jeffektivnosti sistemy razvitija personala organizacii [Methods for analyzing the effectiveness of the organization's personnel development system], Ekonomika i menedzhment innovacionnyh tehnologij [Economics and management of innovative technologies], 11 (2017). URL: https://ekonomika.snauka.ru/2017/11/15439 [23] R. Agarwal, L. Heron, M. Naseh, S. L. Burke, Mentoring Students with Intellectual and Developmental Disabilities: Evaluation of Role‑Specific, Workshops for Mentors and Mentees Journal of Autism and Developmental Disorders (2021) 51:1281–1289. DOI: https://doi.org/10.1007/s10803-020-04599-w [24] R. Y. Tyaningsih et al, The effect of group mentoring learning on student’s creativity in solving partial differential equations problems, Journal of Physics: Conference Series, Seminar Nasional Matematika dan Pendidikan Matematika (SEMADIK), Volume 1778, 2020, 29-30 August 2020, Purwokerto, Indonesia. [25] F. Bate, S. Fyfe, D. Griffiths, K. Russell, C. Skinner, E. Tor, A preliminary evaluation of data- informed mentoring at an Australian medical school, TAPS, 6 (1) (2021) 60-69. DOI: https://doi.org/10.29060/TAPS.2021-6-1/OA2239 [26] S. De Ossorno Garcia, L. Doyle, The mentoring relation as an interpersonal process in EDUCATE: A qualitative case study of mentor–mentee perspectives, Research for All, (2021). URL: https://www.scienceopen.com/document/read?vid=6ef6bc96-68b9-40ec-b76f- a373976a679e [27] A. Leavitt, The Effect of Mentoring for Undergraduate Mentors: A Systematic Review of the Literature, Theses/Capstones/Creative Projects. (2021) 145. URL: https://digitalcommons.unomaha.edu/university_honors_program/145 [28] C. Seery, A. Andres, N. Moore Cherry, S. O’Sullivan, Students as Partners in Peer Mentoring: Expectations, Experiences and Emotions, Innovative Higher Education, 46 (2021) 663–681. DOI: https://doi.org/10.1007/s10755-021-09556-8 [29] F. L. L. Filho, F. V. Andrade de Azevedo, K. L. Augusto, H. S. de Oliveira Silveira, F. A. Martins da Costa, K. P. B. Alves, Mentoring experience in medical residency: challenges and experiences, Experience Report, 45 (03) (2021). DOI: https://doi.org/10.1590/1981-5271v45.3- 20210082.ING [30] K. M. Didur, Suchasni metody otsinky personal [Modern methods of personnel evaluation], Efektyvna ekonomika [Efficient economy], 11 (2011). URL: http://nbuv.gov.ua/UJRN/efek_2011_11_20 [31] H. Y. Iurkevych Suchasni metody otsinky personalu orhanizatsii [Modern methods of evaluating the personnel of the organization], Innovatsiina ekonomika [Innovative economy], 2 (2015) 146- 153. URL: http://nbuv.gov.ua/UJRN/inek_2015_2_28 [32] T. Hilorme, M. Chorna, L. Karpenko, M. Milyavskiy, S. Drobyazko, Innovative model of enterprises personnel incentives evaluation, Academy of Strategic Management Journal, Volume 17, 3 (2018), 1-6. URL:https://www.researchgate.net/profile/Svetlana- Drobyazko/publication/326801240_Innovative_model_of_enterprises_personnel_incentives_eval uation/links/5cde45f0299bf14d959f820c/Innovative-model-of-enterprises-personnel-incentives- evaluation.pdf. [33] V. I. Borshch, Formuvannia mekhanizmu otsiniuvannia efektyvnosti diialnosti upravlinskoho personalu promyslovoho pidpryiemstva [Formation of a mechanism for evaluating the effectiveness of management personnel of an industrial enterprise]. PhD Thesis. Odesa, 2014. 224 p. [34] A. Azarova, V. Puhach, V. Skomarovskyi, Matematychna ta strukturna modeli otsiniuvannia rivnia upravlinnia liudskym kapitalom [Mathematical and structural models for assessing the level of human capital management], Ekonomika ta suspilstvo [Economics and Society], 36 (2022). DOI: https://doi.org/10.32782/2524-0072/2022-36-24 [35] A. M. Grzhibovskij, S. V. Ivanov, M. A. Gorbatova, Sravnenie kolichestvennyh dannyh treh i bolee nezavisimyh vyborok s ispolzovaniem programmnogo obespechenija Statistica i SPSS: parametricheskie i neparametricheskie kriterii [Comparison of quantitative data of three or more independent samples using Statistica and SPSS software: parametric and non-parametric criteria], Nauka i Zdravoohranenie [Science and Healthcare], 4 (2016) 5-37. [36] IT cluster Lviv, dataset, 2021. URL: https://itcluster.lviv.ua/about-us/about-cluster/