=Paper=
{{Paper
|id=Vol-3171/paper104
|storemode=property
|title=Intelligent Systems for Assessment and Development of Personnel: Mentoring Study
|pdfUrl=https://ceur-ws.org/Vol-3171/paper104.pdf
|volume=Vol-3171
|authors=Nestor Shpak,Kateryna Doroshkevych,Zoriana Dvulit,Roman Dzvonyk,Andriy Bosak
|dblpUrl=https://dblp.org/rec/conf/colins/ShpakDDDB22
}}
==Intelligent Systems for Assessment and Development of Personnel: Mentoring Study==
https://ceur-ws.org/Vol-3171/paper104.pdf
Intelligent Systems for Assessment and Development of
Personnel: Mentoring Study
Nestor Shpak1, Kateryna Doroshkevych1, Zoriana Dvulit1, Roman Dzvonyk1 and Andriy
Bosak1
1
Lviv Polytechnic National University, St. Bandery str., 12, Lviv, Ukraine, 79013
Abstract
The article investigates the intellectual systems intended for the evaluation and development
of personnel of the enterprise (intelligent HR systems) and emphasizes the insufficient level
of their development in the conditions of mentoring implementation. To strengthen the
motivational impact on employees and increase the level of enterprise management in
general, the article proposes to develop intelligent systems in the direction of mentoring
assessment, for which an appropriate evaluation procedure has been formed. As part of the
proposed procedure, a scorecard is recommended, which provides for assessing the level of
mentoring processes, identifying the effectiveness of mentoring. To identify the difference in
the level of involvement in the mentoring processes of various structural units of the
enterprise (departments), the article provides posterior comparisons according to the Tukey
HSD. Practical approbation of the order took place in the Lviv IT cluster, where there are
minor differences in the levels of mentoring at enterprises.
Keywords 1
Intelligent systems, personnel, development, assessment, mentoring, Tukey HSD
1. Introduction
As you know, intelligent systems provide a standardized methodological approach to solving
important and rather complex problems and allow you to get consistent and reliable results over time.
[1]. They are widely used in enterprise management for: planning and scheduling of product
development projects, downloading of production capacities of manufacturing enterprise, new product
development and selecting a new product portfolio, evaluation of human resource strategies,
recruitment and promotion, manufacturing design (based on the set of very specialized services that
could be arranged to provide new creative and sustainable processes, etc.), the implementation of the
lean maintenance concept, which allows to increase the operational efficiency of the company's
technical infrastructure, etc. [2-7]. Characteristic of them is the use of artificial intelligence that can
simulate intelligent functions.
As stated in [9], are large-scale application software packages that support business processes and
the flow of information as well as reporting and data analytics in firms as organizations. When
managing personnel, they provide: management of the organizational structure and staffing;
calculation of wages; personnel accounting; time tracking; planning of personnel costs; career
planning and tracking the promotion of personnel in the structure; work with personnel reserve;
recruitment of personnel for vacancies; staff training; attestation systems; competency management.
Thus, a modern intelligent HR system is a complex of technologies that automate and facilitate work
with personnel, ranging from everyday data accounting, ending with strategic decisions on the
COLINS-2022: 6th International Conference on Computational Linguistics and Intelligent Systems, May 12–13, 2022, Gliwice, Poland
EMAIL: nestor.o.shpak@lpnu.ua (N. Shpak); kateryna.o.doroshkevych@lpnu.ua (K. Doroshkevych); zoriana.p.dvulit@lpnu.ua (Z. Dvulit);
roman.y.dzvonyk@lpnu.ua (R. Dzvonyk); andrii.o.bosak@lpnu.ua (A. Bosak).
ORCID: 0000-0003-0620-2458 (N. Shpak); 0000-0003-3966-224X (K. Doroshkevych); 0000-0002-2157-1422 (Z. Dvulit); 0000-0002-
1702-0775 (R. Dzvonyk); 0000-0002-2944-2166 (A. Bosak).
©️ 2022 Copyright for this paper by its authors.
Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
CEUR Workshop Proceedings (CEUR-WS.org)
development of the enterprise. At the same time, mentoring processes remain outside the functionality
of the intelligent system, although they have a significant impact on the development of knowledge in
the enterprise. Let us consider it more carefully. Mentoring as a method of personalized learning has a
significant motivational effect and requires an appropriate level of enterprise costs. The economic
feasibility of activating and applying mentoring at the enterprise as a type of training and personnel
development can be identified in the evaluation process. It, among other things, consists in measuring
the effects, effectiveness, long-term consequences of processes and phenomena, etc. The results
obtained during the evaluation process serve to achieve the goals of developing mentoring programs
[11].
So, in addition to economic efficiency, in the process of evaluation, it is possible to identify the
effectiveness of mentoring, the level of involvement of employees (the intensity of mentoring
influence), its significance, acceptability in view of specific conditions for the implementation of
production and economic activities, etc. Thus, we consider the evaluation of mentoring important for
the implementation of effective management of the enterprise in the conditions of innovative
development. Management decisions on activation, formation of mentoring program, its development,
termination of implementation at the enterprise, etc. depend on its results. That is why the evaluation
process requires systemicity and structure, as well as proper information support. Consider the
processes of assessing mentoring and other methods of training, mastering new professional skills
(approaches, introducing new technologies of work), business qualities of employees, the level of HR
management, etc. On their basis, we will form an architecture that could complement the functionality
of intelligent systems designed for the evaluation and development of enterprise personnel (intelligent
HR systems).
2. Related works
In economic literature, there is no single toolbox designed to evaluate the results of mentoring. For
this purpose, observations, checklists, questionnaires, oral surveys, expert evaluation, analysis of final
works, Color test of relations, Psychometric scale of Likert, Utrecht scale of involvement in the work
of V. Shawfeeli, etc. are used [12-16]. Most of these methods are implemented individually, or are
part of the methods that involve the use of the system of evaluation indicators. For example, the
evaluation stage of mentoring technology according to [17] involves testing, case tasks and
questionnaires to provide feedback that can be made using a competency diagnostic checklist
(example of which is given by the author). This will allow identify the skills of employees received in
the process of mentoring and its effectiveness on the part of the mentor (employees). According to the
recommendations provided by Daniali S., Khortabi F. M., Mohammadbeiki Y., Ilyushnikov K. K.,
Lobova S. V. [18, 19] to assess the level of satisfaction of the head of personnel training should apply
the questionnaire of O. Vetluzhskyh [20]. At the same time, this indicator (the level of satisfaction of
the head with personnel training) is included in the system of indicators and criteria for assessing the
effectiveness of personnel training for railway enterprises. It also includes the following indicators:
implementation of the training plan; staff training costs; loyalty index; use of knowledge gained in
professional activities; increase in productivity; technological violations by fault of employees, etc.
To calculate other indicators, it is provided to calculate using formulas and apply expert assessments.
HR management assessment indicators contain a number of evaluation indicators that are
summarized in three directions: point to the level of organizational management, characterize
technological and educational management [21]. To evaluate each of the areas, the following
indicators are recommended: high-quality staffing; level of staff skillset; staff stability factor;
emotional intelligence; the level of compliance of personnel goals with business goals; degree of
satisfaction with work; level of social tension and conflict; the level of transformation of roles in the
team; employment rate of all roles (according to the Belbin model), etc. The methodology for
calculating indicators provides for expert assessments, mathematical calculation of quantitative
indicators and questionnaires (for example, in the assessment of emotional intelligence). In scientific
work [22] to evaluate the effectiveness of the program development of personnel of the organization,
it is recommended to apply an approach according to which the economic effect will be determined as
the difference between changes in production and training costs. The author recommends various
approaches to assessing the effectiveness of development for the following cases: training is
necessary for the development of a new profession or position in order to further combine positions;
training of employees, which is prescribed by law and controlled by the authorities (in this case, the
economic effect is proposed to be defined as an economic assessment of the consequences of not
conducting this training); assessment of the effect of training non-production personnel; effectiveness
of the personnel development system. Thus, the effectiveness of personnel development systems can
be measured, both in absolute monetary terms and in the form of relative indicators.
To assess the effectiveness of the author's methodology for building an individual trajectory of
self-study, Borisov I.V. proposes to use a six-component model of involvement, which provides for
appropriate engagement indicators and a methodology for their calculation, involving the use of
diagnostic procedures using questionnaires and questionnaires, criteria for their calculation. Further,
to obtain reliable and representative results of the evaluation, the author carried out a statistical
analysis of the obtained results, which provides differential analysis and a posterior comparisons
using the Tukey criterion [12]. Rumi Agarwal,·Laura Heron, Mitra Naseh, Shanna L. Burke used
online platforms (Research Electronic Data Capture (RedCap)), in our research to gather information,
where we were interviewed in the process of training. Then there was a statistical analysis of the
obtained data by conducting paired sample t-tests, followed by correction of Bonferroni to paired
samples of t-tests. Statistical Package for Social Sciences V.20 (SPSS; IBM Corp. 2017) was used by
the authors to analyse quantitative data. [23]. Paired t-tests are also used in evaluating the
effectiveness of the group mentoring model (a learning model that provides group collaboration using
each other's teaching methods, demonstrating and modelling recently acquired knowledge and skills).
In this case, researchers used observation, surveys, testing for information support [24].
In order to collect information about the results of mentoring, online platforms were used by the
Fremantle School of Medicine at the University of Notre Dame (Australia). As part of the study,
university students used their electronic portfolios (supported by Blackboard) to provide feedback in
the process of their studies [25]. Reviews of mentoring programs (scores) collected from the
electronic portfolio of students are further statistically investigated. It is also possible to summarize
the results of employee questionnaires using their personal corporate e-mail boxes, as it is done in
[26].
A thorough study of the mentoring techniques found in the economic literature on the problem of
mentoring undergraduate students [27]. The author studied 80 literary sources in 2013-2020, which
made it possible to summarize the theoretical and conceptual field, data collection methods, obtained
results, etc. In addition to the above methods of collecting data on mentoring, the author noted the
use of individual and focus groups, record logs, written minds of students, ranking (on the persuasion
scale, Likert), mental health testing, etc. In [28] focus groups are combined with the survey and the
use of study diaries.
3. Methods
From the study we conducted, it can be argued that the most common in the practice of
determining the results of personalized learning is to survey mentoring participants using a
standardized questionnaire (questionnaire) with answers posted on the Likert scale. This approach is
used, for the most part, to assess the relevance of the mentoring program and its impact on the
professional activity of the mentee. This approach allows us to identify the impact of mentoring
programs on the activities of relevant groups of employees (doctors, students, etc.). In particular, in
[29] it is established that in addition to the described positive impact, mentoring also serves as
psychosocial support, creating free access between the mentee and the mentor reflected in the
excellent relationship between them. In the conditions of the proper level of implementation of the
enterprise's information systems, information support for the evaluation of mentoring is carried out
using online services, electronic boxes of mentees and mentors, etc., which creates conditions for
automated processing of survey data (questionnaires). This is done by collecting expert opinions on
various aspects of mentoring activities that are not measurable, but are used in calculating indicators.
In the practice of enterprise management, it is common to calculate indicators by which the
effectiveness of mentoring (coaching, mentorship) is revealed. In order to obtain accurate and reliable
results of the assessment, the calculated indicators and results of the questionnaire are subject to
further processing using statistical (differential) analysis. This will ensure the adoption of rational
management decisions based on the results of the implemented study. Approaches to the evaluation of
mentoring in enterprises are summarized in Fig. 1.
Approaches to mentoring assessment
Rating indicators system Questionnaires, surveys,
focus groups, etc.
Statistical and Expert Calculation of
differential assessments indicators
analysis
Figure 1: Approaches to assessing mentoring in enterprises
When evaluating mentoring activities in an organization, the following should be taken into
account. First, the business qualities of the staff are subject to evaluation. According to the
recommendations provided in the [30], the system of assessing the business qualities of personnel
should be carried out in accordance with the directions of the organization and their most significant
property, include standards, criteria of effectiveness, as well as the established procedure for
calculating valuation points, take measures based on the results of the assessment. Based on the
competent approach, the main principles of mentoring assessment are: objectivity; reliability;
predictability; complexity; accessibility and openness; systematic; effectiveness and efficiency.
On the basis of generalization of approaches to the evaluation of mentoring (Fig. 1) in order to
ensure the effectiveness of the assessment, the availability of results for use and compliance with the
principles of mentoring, we propose to implement the following procedure of evaluation: determining
the purpose and objectives of mentoring evaluation; information support of evaluation processes;
selection of methods intended for assessing mentoring in the enterprise; evaluation of mentoring by
implementing the selected method (set of methods); generalization of results and management
decisions (Fig. 2).
Determining the purpose and objectives of mentoring assessment
Information support of mentoring evaluation processes
Selection of methods intended for assessing mentoring at the enterprise
Evaluation of mentoring by implementing the selected method (set of methods)
Generalization of results and management decisions
Figure 2: Recommended procedure for assessing mentoring activities at the enterprise
We will reveal each of the stages of the recommended order.
4. Results and Discussion
The purpose of the mentoring evaluation processes we discussed above, it can be realized by
performing a number of evaluation tasks. We prioritize two main tasks: determining the level of
mentoring processes at the enterprise and identifying the effectiveness of mentoring activities. The
first task is to identify how implemented the mentoring process at the enterprise, the employees
involved in the mentoring process, the plan of personalized training, the mastered budget of
mentoring activities, etc. The second task is to evaluate the results of mentoring activities, covering
the achievement of employees' goals, raising the level of knowledge and their satisfaction with work,
increasing the innovative level of the enterprise, etc.
Information support consists in collecting, processing information that is necessary for the
implementation of assessment tasks. As you know, the effectiveness of this stage is determined by the
use of appropriate information technologies. Since not all aspects of mentoring activities at the
enterprise are subject to quantitative measurement, questionnaires, surveys, examinations, etc. can be
used to assess mentoring. In this case, information support can be developed in the direction of using
online questionnaires, personal accounts of employees, online testing, etc. This will speed up the
processes of collecting, classifying, storing information, etc.
Among the methods intended for assessing mentoring at the enterprise, we recommend using a
system of evaluation indicators that will ensure quantitative assessment, reliability (objectivity),
identify the level of achievement of goals and reserves of mentoring activities and develop directions
for their application, etc. It should be noted that the optimal number of indicators is 5-25 pcs., they
should all be quantitatively measured (in particular, as a result of expert assessments) and criterion
(the obtained value of the indicator should indicate the level of achievement of the corresponding
mentoring goal). We recommend a system of indicators covering 2 areas of evaluation: the level of
mentoring processes in the enterprise, the effectiveness of the company's mentoring (Table 1).
Table 1
The system of indicators intended for the evaluation of mentoring activities in enterprises
№ Indicators Essence Calculation Criteria values
1 2 3 4 5
Assessment of the level of mentoring processes in the enterprise
1 Mentoring Plan The ratio of the actual 𝑅𝑒𝑎 [0; 0,55] – non-fulfilment of
𝑀𝑝𝑒𝑙 =
Execution Level number of employees 𝑅𝑒𝑎𝑝 the plan; [0,56; 0,75] -
(𝑀𝑝𝑒𝑙 ) involved in the mentoring partial implementation of
process (𝑅𝑒𝑎 ) to planned the plan; [0,76; 1] - execute
(𝑅𝑒𝑝 ) the plan
2 The level of Indicates the ratio of 𝐶𝑡𝑒𝑓 [0; 0,55] – minimum cost
𝐿𝑡𝑒𝑒 =
expenses for actual costs made to 𝐶𝑡𝑒𝑝 level; [0,56; 0,75] – average
mentoring mentoring and other types cost level;[0,76; 1] –
activities and of training in the optimal cost level
other types of enterprise (𝐶𝑡𝑒𝑓 ) to
training of planned, which should be
employees 5% of the remuneration
(𝐿𝑡𝑒𝑒 ) fund (𝐶𝑡𝑒𝑝 )
3 Employee It is calculated similarly to 𝐼𝑒𝑙 =𝑆𝑠𝑚𝑒 − [0; 0,55] – mentoring
loyalty index to the Employee Loyalty 𝑆𝑐𝑚𝑒 program should be
mentoring (𝐼𝑒𝑙 ) Index (eNPS), which replaced; [0,56; 0,75] –
implies a difference mentoring program should
between the share of be adjusted; [0,76; 1] –
supporters (𝑆𝑠𝑚𝑒 ) and mentoring program meets
mentoring critics among the goals and objectives of
employees (𝑆𝑐𝑚𝑒 ) the enterprise
1 2 3 4 5
4 Managers' It is calculated similarly to 𝐼𝑚𝑙 =𝑆𝑠𝑚𝑚 − [0; 0,55] – managers are
loyalty index to the Employee Loyalty 𝑆𝑐𝑚𝑚 completely dissatisfied with
employee Index (eNPS), which the program; [0,56; 0,75] –
mentoring (𝐼𝑚𝑙 ) provides for the difference managers are partially
between the share of satisfied with the
supporters (𝑆𝑠𝑚𝑚 ) and mentoring program;
mentoring critics (𝑆𝑐𝑚𝑚 ) [0,76; 1] – managers are
among managers whose loyal to the mentoring
staff has been trained program
Evaluation of the effectiveness of mentoring in the enterprise
5 The level of The ratio of the number of 𝑁𝑠𝑝 [0; 0,55] – low stability of
𝐿𝑠𝑐𝑝 =
stability of the employees who work at 𝑇𝑛𝑒 the company's personnel;
company's the enterprise for more [0,56; 0,75] – average
personnel than a year (𝑁𝑠𝑝 ) to the stability of the company's
(𝐿𝑠𝑐𝑝 ) total number of employees personnel; [0,76; 1] –
of the enterprise (𝑇𝑛𝑒 ) stable staff
6 Level of Ratio of the number of 𝑁 𝑒𝑖𝑞 [0; 0,55] – low level of
𝐿𝑝𝑑𝑒 =
professional employees who improved 𝑇𝑛𝑒 professional development
development of their qualifications in the of employees; [0,56; 0,75] –
employees reporting period (𝑁𝑒𝑖𝑞 ) to average level of
(𝐿𝑝𝑑𝑒 ) the total number of professional development
employees of the of employees; [0,76; 1] –
enterprise (𝑇𝑛𝑒 ) high level of professional
development of employees
7 The level of The ratio of the number of 𝑁𝑒𝑖𝑖 [0; 0,55] – low level; [0,56;
𝐿𝑒𝑖𝑖 =
involvement of employees involved in 𝑇𝑛𝑒 0,75] – average level; [0,76;
personnel in innovation activities (𝑁𝑒𝑖𝑖 ) 1] – high level of
innovation to the total number of involvement of employees
(𝐿𝑒𝑖𝑖 ) employees (𝑇𝑛𝑒 ) in innovation activities
8 The level of The ratio of the number of 𝑁𝑗𝑎𝑞 [0; 0,55] – low level; [0,56;
𝐿𝑞𝑠𝑒 =
quantitative employees who took jobs 𝑇𝑛𝑒 0,75] – average level; [0,76;
staffing of the of appropriate 1] – high level of staffing of
enterprise qualification (𝑁𝑗𝑎𝑞 ) to the the state of the enterprise
states (𝐿𝑞𝑠𝑒 ) total number of employees
of the enterprise (𝑇𝑛𝑒 )
9 Average goal Indicates the level of Expert 0 – employee's goals do not
match rank compliance of personnel evaluation meet the goals of the
(𝐴𝑔𝑚𝑟 ) goals with the goals of the according to enterprise; 0,5 – partial
enterprise the specified compliance with the goals
criteria of the employee and the
enterprise; 1 – employee's
goals fully meet the goals of
the enterprise
10 The level of Indicates the ratio of 𝑅𝑝𝑓 [0; 0,55] – the level of
𝐿𝑖𝑝 =
increase in actual productivity (𝑅𝑝𝑓 ) 𝑅𝑝𝑝 increase in production is
productivity to the planned one, which minimal; [0,56; 0,75] –
(production) at should be 5% more than average level; [0,76; 1] –
the enterprise the productivity in the the level of increase in
(𝐿𝑖𝑝 ) base period (𝑅𝑝𝑝 ) production is optimal
The information base for calculating indicators is the data of analytical and management
accounting of enterprises, as well as expert assessments. On their basis, indicators should be
calculated, as well as generalizations should be carried out (for example, determining the average
level as provided for in calculating the average rank of compliance with goals, etc.). In order to
summarize the results of the assessment, we propose to identify the integral level of indicators (𝐼𝑚𝑎 ),
taking into account their weight. This is ensured in the process of using the factorial method by the
formula:
𝐼𝑚𝑎 = 𝑀𝑝𝑒𝑙 × 𝑘𝑝𝑒𝑙 + 𝐿𝑡𝑒𝑒 × 𝑘𝑡𝑒𝑒 + 𝐼𝑒𝑙 × 𝑘𝑒𝑙 + 𝐼𝑚𝑙 × 𝑘𝑚𝑙 + 𝐿𝑠𝑐𝑝 × 𝑘𝑠𝑐𝑝 +
+𝐿𝑝𝑑𝑒 × 𝑘𝑝𝑑𝑒 + 𝐿𝑒𝑖𝑖 × 𝑘𝑒𝑖𝑖 + 𝐿𝑞𝑠𝑒 × 𝑘𝑞𝑠𝑒 + 𝐴𝑔𝑚𝑟 × 𝑘𝑔𝑚𝑟 + 𝐿𝑖𝑝 × 𝑘𝑖𝑝 , (1)
where 𝑘𝑖 – weight of indicators intended for the evaluation of mentoring activities in enterprises.
∑10
𝑖=1 𝑘𝑖 = 1.
According to the recommendations provided in the economic literature on identifying the level of
involvement of individuals in the learning process, the results obtained should be checked for
statistical effects that can be evaluated using two-factual dispersion analysis of all components of
involvement in mentoring processes [12]. To identify the difference in the level of involvement of
various structural units of the enterprise (departments) in the mentoring processes. In order to identify
differences in the statistically significant studied criteria of the groups (departments) and posterior
comparisons should be made according to the Tukey criterion [31-35]. As a result, we will find out
which departments (enterprises) differ in the level of implementation of individual mentoring
components that characterize the recommended indicators (Table 1).
4. Experimental
We will evaluate mentoring activities at Lviv IT cluster enterprises. Lviv IT Cluster is a
community of IT companies, government and education that have taken the responsibility to work on
the development of the industry and the region through education, industry promo (city, region and
companies) and infrastructure [36]. Cluster participants are companies of different sizes and with
different needs, most of them have experience in mentoring activities that should be evaluated. Thus,
the presence of different enterprises in the cluster allows you to evaluate individually by enterprises
and identify the level of involvement of employees in mentoring activities in the cluster. To do this,
we calculate the indicators intended for assessing mentoring at the enterprises of the cluster (Table 2).
Table 2
Values of indicators intended for evaluation of mentoring activities at Lviv IT cluster enterprises
№ Indicators Value in enterprises
#1 #2 #3 #4
Assessment of the level of mentoring processes in the enterprise
1 Mentoring Plan Execution Level (𝑀𝑝𝑒𝑙 ) 0,82 0,84 0,76 0,8
2 The level of expenses for mentoring activities and other types of 0,9 0,86 0,92 0,95
training of employees (𝐿𝑡𝑒𝑒 )
3 Employee loyalty index to mentoring (𝐼𝑒𝑙 ) 0,56 0,82 0,73 0,58
4 Managers' loyalty index to employee mentoring (𝐼𝑚𝑙 ) 0,72 0,68 0,59 0,8
Evaluation of the effectiveness of mentoring in the enterprise
5 The level of stability of the company's personnel (𝐿𝑠𝑐𝑝 ) 0,53 0,75 0,48 0,6
6 Level of professional development of employees (𝐿𝑝𝑑𝑒 ) 0,86 0,72 0,76 0,84
7 The level of involvement of personnel in innovation ( 𝐿𝑒𝑖𝑖 ) 0,76 0,78 0,8 0,78
8 The level of quantitative staffing of the enterprise states (𝐿𝑞𝑠𝑒 ) 0,86 0,9 0,79 0,82
9 Average goal match rank (𝐴𝑔𝑚𝑟 ) 1 1 0,5 1
10 The level of increase in productivity (production) at the enterprise 0,63 0,5 0,76 0,89
(𝐿𝑖𝑝 )
Lviv IT Cluster mission: "In the future, we see Lviv as a world-class high technology center. And
our mission is to contribute to this as much as possible." The cluster includes companies, educational
institutions, local authorities and BPO participants. According to this we will carry out an expert
assessment of the weight of each of the indicators included in the recommended system (Table 1).
This will reveal the level of mentoring activity at each of the enterprises of the Lviv IT cluster. The
results are summarized in Table. 3.
Table 3
Weight of indicators intended for evaluation of mentoring activities at Lviv IT cluster enterprises
№ Indicators Weight
Assessment of the level of mentoring processes in the enterprise
1 Mentoring Plan Execution Level (𝑀𝑝𝑒𝑙 ) 0,13
2 The level of expenses for mentoring activities and other types of training of
0,03
employees (𝐿𝑡𝑒𝑒 )
3 Employee loyalty index to mentoring (𝐼𝑒𝑙 ) 0,16
4 Managers' loyalty index to employee mentoring (𝐼𝑚𝑙 ) 0,05
Evaluation of the effectiveness of mentoring in the enterprise
5 The level of stability of the company's personnel (𝐿𝑠𝑐𝑝 ) 0,13
6 Level of professional development of employees (𝐿𝑝𝑑𝑒 ) 0,03
7 The level of involvement of personnel in innovation (𝐿𝑒𝑖𝑖 ) 0,16
8 The level of quantitative staffing of the enterprise states (𝐿𝑞𝑠𝑒 ) 0,05
9 Average goal match rank (𝐴𝑔𝑚𝑟 ) 0,1
10 The level of increase in productivity (production) at the enterprise (𝐿𝑖𝑝 ) 0,12
We base the obtained values of meaning of indicators (Table 2) and weight for enterprises of Lviv
IT cluster (Table 3) in the equation (1). As a result, the level of mentoring activity (𝐼𝑚𝑎 ) of each of the
enterprises was obtained: for the enterprise 1 𝐼𝑚𝑎 =0,74; for the enterprise 2 𝐼𝑚𝑎 =0,76; for the
enterprise 3 𝐼𝑚𝑎 =0,67; for the enterprise 4 𝐼𝑚𝑎 =0,79.
Next, we will compare the levels of mentoring of enterprises in the Lviv IT cluster using the online
service (https://astatsa.com/OneWay_Anova_with_TukeyHSD/), which conducts posterior
comparisons according to the Tukey criterion. The results are summarized in Table. 4.
Table 4
Posterior comparisons of the level of mentoring activity at the enterprises of Lviv IT cluster according
to the Tukey criterion
Tukey HSD Tukey HSD Tukey HSD
Treatments pair
Q statistic p-value inference
Enterprise 1 and insignificant
0.1636 0.8999947
Enterprise 2
Enterprise 1 and insignificant
0.4658 0.8999947
Enterprise 3
Enterprise 1 and insignificant
0.4127 0.8999947
Enterprise 4
Enterprise 2 and insignificant
0.6294 0.8999947
Enterprise 3
Enterprise 2 and insignificant
0.2490 0.8999947
Enterprise 4
Enterprise 3 and insignificant
0.8785 0.8999947
Enterprise 4
As a result of the calculations, we can affirm insignificant differences in the levels of mentoring
for enterprises of the Lviv IT Cluster and a sufficient level of mentoring activities of the cluster. This
is also evidenced by the data obtained as a result of the aggregation of indicators and the
determination of the integral level of mentoring activities.
5. Conclusions
The article substantiated the importance of assessing mentoring as a method of personalized
learning, which has a significant motivational impact on the employees of the enterprise and the
effectiveness of their activities. To ensure this, approaches to the evaluation of mentoring at
enterprises have been studied and summarized, as a result of which the procedure for evaluating
mentoring activities and the scorecard are recommended. The recommended procedure contains the
stages of mentoring assessment, and the indicators are grouped into two areas of evaluation:
assessment of the level of mentoring processes in the enterprise (level of implementation of the
mentoring plan, the level of costs for mentoring activities and other types of training of employees,
the index of loyalty of employees to mentoring, the index of loyalty of managers to mentoring
employees), determining the effectiveness of mentoring the enterprise (level of stability of the
company's personnel, the level of professional development of employees, the level of involvement of
personnel in innovation, the level of quantitative complexity of the states of the enterprise, the
average rank of compliance with goals, the level of increase in productivity (production) in the
enterprise). The recommended procedure provides for summarizing indicators and calculating the
integral level of mentoring activities and identifying differences in the level of involvement in the
mentoring processes of various structural units of the enterprise (departments), for which a posterior
comparisons are provided according to the Tukey criterion. Practical approbation of the order took
place on the basis of the Lviv IT Cluster, where there are minor differences in the levels of mentoring
at enterprises.
Further research should explore the possibility of combining the developed order of mentoring
evaluation with intelligent HR systems in the enterprise (DeloPro, SAP R/3, Baan, Oracle
Applications, etc.).
6. References
[1] T. A. Byrd, R. D. Hauser, Expert systems in production and operations management: research
directions in assessing overall impact, Int. J. Prod. Res., Volume 29, (1991) 2471-2482.
[2] M. Relich, W. Muszyński, The Use of Intelligent Systems for Planning and Scheduling of
Product Development Projects, Procedia Computer Science, Volume 35, (2014) 1586-1595.
[3] M. Relich, K. Bzdyra, Estimating new product success with the use of intelligent systems,
Foundations of Management, Volume 6, 2 (2014) 7-20.
[4] Y. Wang, L. Li, L. Yang, Intelligent Human Resource Planning System in a Large Petrochemical
Enterprise, IEEE Intelligent Systems, Volume 28, 4 (2013) 102-106. DOI:
10.1109/MIS.2013.112.
[5] J. R. Silva, New Trends in Manufacturing: Converging to Service and Intelligent Systems, IFAC
Proceedings, Volume 47, 3 (2014) 2628-2633, URL: https://doi.org/10.3182/20140824-6-ZA-
1003.02823.
[6] K. Antosz, L. Pasko, A. Gola, The Use of Intelligent Systems to Support the Decision-Making
Process in Lean Maintenance Management, IFAC-Papers, Volume 52, 10 (2019) 148-153. URL:
https://doi.org/10.1016/j.ifacol.2019.10.037.
[7] N. Shpak, M. Odrekhivskyi, K. Doroshkevych, W. Sroka, Simulation of Innovative Systems
under Industry 4.0 Conditions, Social Science 8 (7) (2019) 202. DOI: 10.3390/socsci8070202.
[8] N. Shpak, N. Podolchak, V. Karkovska, W. Sroka. The Influence of Age Factors on the Reform
of the Public Service of Ukraine, Central European Journal of Public Policy, 13(2), (2019) 40–
52. DOI: 10.2478/cejpp-2019-0006.
[9] A. Mouzoune, Contribution to Enterprise Intelligent Systems Architecture: Assumptions,
Expectations and a Proposal, Intelligent Information Management, Vollume 4, 6 (2012) 1-8.
DOI:10.4236/iim.2012.46035.
[10] N. Shpak, K. Doroshkevych, Y. Shpak, I. Salata, M. Sharko, Strategy and Tactics of
International Digitalization and Intellectualization of Economic Relations, CEUR Workshop
Proceedings: Proceedings of 5th International Conference on Computational Linguistics and
Intelligent Systems (COLINS-2021), Volume 2870 (2021) 1477-1487. URL: http://ceur-
ws.org/Vol-2870/paper109.pdf.
[11] A. J. Hobson, B. Maxwell, K. Káplár-Kodácsy, The Role of the Mentoring, Programme
Coordinator: A Rapid Evidence Review Education & Training Foundation, University of
Brighton and Sheffield Hallam University, (05) 2021. URL: https://www.et-
foundation.co.uk/wp-
content/uploads/2021/05/ETF_MCoord_Report_UoB_SHU_29_April_2021_Final.pdf.
[12] D. I. Martsenkovskyi, Vikovyi polimorfizm posttravmatychnykh stresovykh rozladiv ta depresii
u ditei vnutrishno peremishchenykh osib z okupovanykh terytorii [Age polymorphism of post-
traumatic stress disorders and depression in children of internally displaced persons from the
occupied territories], Arkhiv psykhiatrii [Archives of Psychiatry], Volume 24, 1 (2018) 15-20.
URL: http://nbuv.gov.ua/UJRN/apsuh_2018_24_1_4.
[13] J. Batut, M. Kvaskoff, M. C. Morris, When mentoring matters: a French mentoring program for
women in science, Nature Biotechnology, 39 (2021) 776–779. URL:
https://www.nature.com/articles/s41587-021-00951-2.
[14] A. G. Opsahl, C. Townsend, Mentoring to engage diverse undergraduate nursing students in
honors research, Nurs Forum, 56:19 (2021). DOI: https://doi.org/10.1111/nuf.12503.
[15] D. Games, The Effectiveness of Mentoring and Coaching in Teaching Business Planning: The
Case of Universitas Andalas, Advances in Social Science, Education and Humanities Research,
Proceedings of the 3rd International Conference on Educational Development and Quality
Assurance (ICED-QA 2020), 506 (2020) 1-5.
[16] S. Resnjanskij, J. Ruhose, S. Wiederhold, L. Woessmann, Mentoring Improves the Labor-Market
Prospects of Disadvantaged Adolescents, CESifo Forum, Volume 22, 4 (2021) 38-73. URL:
https://www.cesifo.org/DocDL/CESifo-forum-2021-4-resnjanskij-etal-Mentoring-Improves-the-
Labor-Market-Prospects-of-Disadvantaged-Adolescents.pdf.
[17] G. I. Egorova, Tehnologicheskij diskurs povyshenija urovnja professionalnyh kompetencij
pedagoga [Technological discourse of increasing the level of professional competencies of a
teacher], Humanities researches of the Central Russia, 1 (18) (2021) 45-63. URL:
https://cyberleninka.ru/article/n/tehnologicheskiy-diskurs-povysheniya-urovnya-professionalnyh-
kompetentsiy-pedagoga/viewer.
[18] S. Daniali, F. M. Khortabi, Y. Mohammadbeiki, Innovative approach for evaluating training
effectiveness, Proceedings of the 32nd International Business Information Management
Association Conference, IBIMA 2018 – Vision 2020: Sustainable Economic Development and
Application of Innovation Management from Regional expansion to Global Growth, 32 (2018)
818-830.
[19] K. K. Ilyushnikov, S. V. Lobova, Evaluation model for effectiveness of corporate training based
on key performance indicators and detailed metrics, Espacios, Volume 40, 1 (2019), 29-37.
URL:http://www.revistaespacios.com/ a19v40n01/a19v40n01p29.pdf
[20] E. N. Vetluzhskih, Obuchaem – ocenivaem. Modeli kriterialnoj ocenki rezultatov obuchenija
[Models for Criteria-Based Assessment of Learning Outcomes], Spravochnik po upravleniju
personalom [Human Resources Handbook], 2 (2005) 10-18.
[21] T. L. Bezrukova et al, Optimization model of motivational process as a basis of management
decisions, IOP Conf. Ser.: Mater. Sci. Eng, 483 012025 (2019), DOI:10.1088/1757-
899X/483/1/012025.
[22] E. A. Belalova, Metody analiza jeffektivnosti sistemy razvitija personala organizacii [Methods
for analyzing the effectiveness of the organization's personnel development system], Ekonomika
i menedzhment innovacionnyh tehnologij [Economics and management of innovative
technologies], 11 (2017). URL: https://ekonomika.snauka.ru/2017/11/15439
[23] R. Agarwal, L. Heron, M. Naseh, S. L. Burke, Mentoring Students with Intellectual and
Developmental Disabilities: Evaluation of Role‑Specific, Workshops for Mentors and Mentees
Journal of Autism and Developmental Disorders (2021) 51:1281–1289. DOI:
https://doi.org/10.1007/s10803-020-04599-w
[24] R. Y. Tyaningsih et al, The effect of group mentoring learning on student’s creativity in solving
partial differential equations problems, Journal of Physics: Conference Series, Seminar Nasional
Matematika dan Pendidikan Matematika (SEMADIK), Volume 1778, 2020, 29-30 August 2020,
Purwokerto, Indonesia.
[25] F. Bate, S. Fyfe, D. Griffiths, K. Russell, C. Skinner, E. Tor, A preliminary evaluation of data-
informed mentoring at an Australian medical school, TAPS, 6 (1) (2021) 60-69. DOI:
https://doi.org/10.29060/TAPS.2021-6-1/OA2239
[26] S. De Ossorno Garcia, L. Doyle, The mentoring relation as an interpersonal process in
EDUCATE: A qualitative case study of mentor–mentee perspectives, Research for All, (2021).
URL: https://www.scienceopen.com/document/read?vid=6ef6bc96-68b9-40ec-b76f-
a373976a679e
[27] A. Leavitt, The Effect of Mentoring for Undergraduate Mentors: A Systematic Review of the
Literature, Theses/Capstones/Creative Projects. (2021) 145. URL:
https://digitalcommons.unomaha.edu/university_honors_program/145
[28] C. Seery, A. Andres, N. Moore Cherry, S. O’Sullivan, Students as Partners in Peer Mentoring:
Expectations, Experiences and Emotions, Innovative Higher Education, 46 (2021) 663–681.
DOI: https://doi.org/10.1007/s10755-021-09556-8
[29] F. L. L. Filho, F. V. Andrade de Azevedo, K. L. Augusto, H. S. de Oliveira Silveira, F. A.
Martins da Costa, K. P. B. Alves, Mentoring experience in medical residency: challenges and
experiences, Experience Report, 45 (03) (2021). DOI: https://doi.org/10.1590/1981-5271v45.3-
20210082.ING
[30] K. M. Didur, Suchasni metody otsinky personal [Modern methods of personnel evaluation],
Efektyvna ekonomika [Efficient economy], 11 (2011). URL:
http://nbuv.gov.ua/UJRN/efek_2011_11_20
[31] H. Y. Iurkevych Suchasni metody otsinky personalu orhanizatsii [Modern methods of evaluating
the personnel of the organization], Innovatsiina ekonomika [Innovative economy], 2 (2015) 146-
153. URL: http://nbuv.gov.ua/UJRN/inek_2015_2_28
[32] T. Hilorme, M. Chorna, L. Karpenko, M. Milyavskiy, S. Drobyazko, Innovative model of
enterprises personnel incentives evaluation, Academy of Strategic Management Journal, Volume
17, 3 (2018), 1-6. URL:https://www.researchgate.net/profile/Svetlana-
Drobyazko/publication/326801240_Innovative_model_of_enterprises_personnel_incentives_eval
uation/links/5cde45f0299bf14d959f820c/Innovative-model-of-enterprises-personnel-incentives-
evaluation.pdf.
[33] V. I. Borshch, Formuvannia mekhanizmu otsiniuvannia efektyvnosti diialnosti upravlinskoho
personalu promyslovoho pidpryiemstva [Formation of a mechanism for evaluating the
effectiveness of management personnel of an industrial enterprise]. PhD Thesis. Odesa, 2014.
224 p.
[34] A. Azarova, V. Puhach, V. Skomarovskyi, Matematychna ta strukturna modeli otsiniuvannia
rivnia upravlinnia liudskym kapitalom [Mathematical and structural models for assessing the
level of human capital management], Ekonomika ta suspilstvo [Economics and Society], 36
(2022). DOI: https://doi.org/10.32782/2524-0072/2022-36-24
[35] A. M. Grzhibovskij, S. V. Ivanov, M. A. Gorbatova, Sravnenie kolichestvennyh dannyh treh i
bolee nezavisimyh vyborok s ispolzovaniem programmnogo obespechenija Statistica i SPSS:
parametricheskie i neparametricheskie kriterii [Comparison of quantitative data of three or more
independent samples using Statistica and SPSS software: parametric and non-parametric
criteria], Nauka i Zdravoohranenie [Science and Healthcare], 4 (2016) 5-37.
[36] IT cluster Lviv, dataset, 2021. URL: https://itcluster.lviv.ua/about-us/about-cluster/