The article investigates the intellectual systems intended for the evaluation and development of personnel of the enterprise (intelligent HR systems) and emphasizes the insufficient level of their development in the conditions of mentoring implementation. To strengthen the motivational impact on employees and increase the level of enterprise management in general, the article proposes to develop intelligent systems in the direction of mentoring assessment, for which an appropriate evaluation procedure has been formed. As part of the proposed procedure, a scorecard is recommended, which provides for assessing the level of mentoring processes, identifying the effectiveness of mentoring. To identify the difference in the level of involvement in the mentoring processes of various structural units of the enterprise (departments), the article provides posterior comparisons according to the Tukey HSD. Practical approbation of the order took place in the Lviv IT cluster, where there are minor differences in the levels of mentoring at enterprises.
As you know, intelligent systems provide a standardized methodological approach to solving
important and rather complex problems and allow you to get consistent and reliable results over time.
[
product portfolio, evaluation of human resource strategies,
recruitment and promotion, manufacturing design (based on the set of very specialized services that
could be arranged to provide new creative and sustainable processes, etc.), the implementation of the
lean maintenance concept, which allows to increase the operational efficiency of the company's
technical infrastructure, etc. [
As stated in [9], are large-scale application software packages that support business processes and the flow of information as well as reporting and data analytics in firms as organizations. When managing personnel, they provide: management of the organizational structure and staffing; calculation of wages; personnel accounting; time tracking; planning of personnel costs; career planning and tracking the promotion of personnel in the structure; work with personnel reserve; recruitment of personnel for vacancies; staff training; attestation systems; competency management. Thus, a modern intelligent HR system is a complex of technologies that automate and facilitate work with personnel, ranging from everyday data accounting, ending with strategic decisions on the
2022 Copyright for this paper by its authors. development of the enterprise. At the same time, mentoring processes remain outside the functionality of the intelligent system, although they have a significant impact on the development of knowledge in the enterprise. Let us consider it more carefully. Mentoring as a method of personalized learning has a significant motivational effect and requires an appropriate level of enterprise costs. The economic feasibility of activating and applying mentoring at the enterprise as a type of training and personnel development can be identified in the evaluation process. It, among other things, consists in measuring the effects, effectiveness, long-term consequences of processes and phenomena, etc. The results obtained during the evaluation process serve to achieve the goals of developing mentoring programs [11].
So, in addition to economic efficiency, in the process of evaluation, it is possible to identify the effectiveness of mentoring, the level of involvement of employees (the intensity of mentoring influence), its significance, acceptability in view of specific conditions for the implementation of production and economic activities, etc. Thus, we consider the evaluation of mentoring important for the implementation of effective management of the enterprise in the conditions of innovative development. Management decisions on activation, formation of mentoring program, its development, termination of implementation at the enterprise, etc. depend on its results. That is why the evaluation process requires systemicity and structure, as well as proper information support. Consider the processes of assessing mentoring and other methods of training, mastering new professional skills (approaches, introducing new technologies of work), business qualities of employees, the level of HR management, etc. On their basis, we will form an architecture that could complement the functionality of intelligent systems designed for the evaluation and development of enterprise personnel (intelligent HR systems).
In economic literature, there is no single toolbox designed to evaluate the results of mentoring. For this purpose, observations, checklists, questionnaires, oral surveys, expert evaluation, analysis of final works, Color test of relations, Psychometric scale of Likert, Utrecht scale of involvement in the work of V. Shawfeeli, etc. are used [12-16]. Most of these methods are implemented individually, or are part of the methods that involve the use of the system of evaluation indicators. For example, the evaluation stage of mentoring technology according to [17] involves testing, case tasks and questionnaires to provide feedback that can be made using a competency diagnostic checklist (example of which is given by the author). This will allow identify the skills of employees received in the process of mentoring and its effectiveness on the part of the mentor (employees). According to the recommendations provided by Daniali S., Khortabi F. M., Mohammadbeiki Y., Ilyushnikov K. K., Lobova S. V. [18, 19] to assess the level of satisfaction of the head of personnel training should apply the questionnaire of O. Vetluzhskyh [20]. At the same time, this indicator (the level of satisfaction of the head with personnel training) is included in the system of indicators and criteria for assessing the effectiveness of personnel training for railway enterprises. It also includes the following indicators: implementation of the training plan; staff training costs; loyalty index; use of knowledge gained in professional activities; increase in productivity; technological violations by fault of employees, etc. To calculate other indicators, it is provided to calculate using formulas and apply expert assessments.
HR management assessment indicators contain a number of evaluation indicators that are summarized in three directions: point to the level of organizational management, characterize technological and educational management [21]. To evaluate each of the areas, the following indicators are recommended: high-quality staffing; level of staff skillset; staff stability factor; emotional intelligence; the level of compliance of personnel goals with business goals; degree of satisfaction with work; level of social tension and conflict; the level of transformation of roles in the team; employment rate of all roles (according to the Belbin model), etc. The methodology for calculating indicators provides for expert assessments, mathematical calculation of quantitative indicators and questionnaires (for example, in the assessment of emotional intelligence). In scientific work [22] to evaluate the effectiveness of the program development of personnel of the organization, it is recommended to apply an approach according to which the economic effect will be determined as the difference between changes in production and training costs. The author recommends various approaches to assessing the effectiveness of development for the following cases: training is necessary for the development of a new profession or position in order to further combine positions; training of employees, which is prescribed by law and controlled by the authorities (in this case, the economic effect is proposed to be defined as an economic assessment of the consequences of not conducting this training); assessment of the effect of training non-production personnel; effectiveness of the personnel development system. Thus, the effectiveness of personnel development systems can be measured, both in absolute monetary terms and in the form of relative indicators.
To assess the effectiveness of the author's methodology for building an individual trajectory of self-study, Borisov I.V. proposes to use a six-component model of involvement, which provides for appropriate engagement indicators and a methodology for their calculation, involving the use of diagnostic procedures using questionnaires and questionnaires, criteria for their calculation. Further, to obtain reliable and representative results of the evaluation, the author carried out a statistical analysis of the obtained results, which provides differential analysis and a posterior comparisons using the Tukey criterion [12]. Rumi Agarwal,·Laura Heron, Mitra Naseh, Shanna L. Burke used online platforms (Research Electronic Data Capture (RedCap)), in our research to gather information, where we were interviewed in the process of training. Then there was a statistical analysis of the obtained data by conducting paired sample t-tests, followed by correction of Bonferroni to paired samples of t-tests. Statistical Package for Social Sciences V.20 (SPSS; IBM Corp. 2017) was used by the authors to analyse quantitative data. [23]. Paired t-tests are also used in evaluating the effectiveness of the group mentoring model (a learning model that provides group collaboration using each other's teaching methods, demonstrating and modelling recently acquired knowledge and skills). In this case, researchers used observation, surveys, testing for information support [24].
In order to collect information about the results of mentoring, online platforms were used by the Fremantle School of Medicine at the University of Notre Dame (Australia). As part of the study, university students used their electronic portfolios (supported by Blackboard) to provide feedback in the process of their studies [25]. Reviews of mentoring programs (scores) collected from the electronic portfolio of students are further statistically investigated. It is also possible to summarize the results of employee questionnaires using their personal corporate e-mail boxes, as it is done in [26].
A thorough study of the mentoring techniques found in the economic literature on the problem of mentoring undergraduate students [27]. The author studied 80 literary sources in 2013-2020, which made it possible to summarize the theoretical and conceptual field, data collection methods, obtained results, etc. In addition to the above methods of collecting data on mentoring, the author noted the use of individual and focus groups, record logs, written minds of students, ranking (on the persuasion scale, Likert), mental health testing, etc. In [28] focus groups are combined with the survey and the use of study diaries.
From the study we conducted, it can be argued that the most common in the practice of determining the results of personalized learning is to survey mentoring participants using a standardized questionnaire (questionnaire) with answers posted on the Likert scale. This approach is used, for the most part, to assess the relevance of the mentoring program and its impact on the professional activity of the mentee. This approach allows us to identify the impact of mentoring programs on the activities of relevant groups of employees (doctors, students, etc.). In particular, in [29] it is established that in addition to the described positive impact, mentoring also serves as psychosocial support, creating free access between the mentee and the mentor reflected in the excellent relationship between them. In the conditions of the proper level of implementation of the enterprise's information systems, information support for the evaluation of mentoring is carried out using online services, electronic boxes of mentees and mentors, etc., which creates conditions for automated processing of survey data (questionnaires). This is done by collecting expert opinions on various aspects of mentoring activities that are not measurable, but are used in calculating indicators. In the practice of enterprise management, it is common to calculate indicators by which the effectiveness of mentoring (coaching, mentorship) is revealed. In order to obtain accurate and reliable results of the assessment, the calculated indicators and results of the questionnaire are subject to further processing using statistical (differential) analysis. This will ensure the adoption of rational management decisions based on the results of the implemented study. Approaches to the evaluation of mentoring in enterprises are summarized in Fig. 1.
assessments
When evaluating mentoring activities in an organization, the following should be taken into account. First, the business qualities of the staff are subject to evaluation. According to the recommendations provided in the [30], the system of assessing the business qualities of personnel should be carried out in accordance with the directions of the organization and their most significant property, include standards, criteria of effectiveness, as well as the established procedure for calculating valuation points, take measures based on the results of the assessment. Based on the competent approach, the main principles of mentoring assessment are: objectivity; reliability; predictability; complexity; accessibility and openness; systematic; effectiveness and efficiency.
On the basis of generalization of approaches to the evaluation of mentoring (Fig. 1) in order to ensure the effectiveness of the assessment, the availability of results for use and compliance with the principles of mentoring, we propose to implement the following procedure of evaluation: determining the purpose and objectives of mentoring evaluation; information support of evaluation processes; selection of methods intended for assessing mentoring in the enterprise; evaluation of mentoring by implementing the selected method (set of methods); generalization of results and management decisions (Fig. 2).
We will reveal each of the stages of the recommended order.
The purpose of the mentoring evaluation processes we discussed above, it can be realized by performing a number of evaluation tasks. We prioritize two main tasks: determining the level of mentoring processes at the enterprise and identifying the effectiveness of mentoring activities. The first task is to identify how implemented the mentoring process at the enterprise, the employees involved in the
mentoring process, the plan of personalized training, the mastered budget of mentoring activities, etc. The second task is to evaluate the results of mentoring activities, covering the achievement of employees' goals, raising the level of knowledge and their satisfaction with work, increasing the innovative level of the enterprise, etc.
Information support consists in collecting, processing information that is necessary for the implementation of assessment tasks. As you know, the effectiveness of this stage is determined by the use of appropriate information technologies. Since not all aspects of mentoring activities at the enterprise are subject to quantitative measurement, questionnaires, surveys, examinations, etc. can be used to assess mentoring. In this case, information support can be developed in the direction of using online questionnaires, personal accounts of employees, online testing, etc. This will speed up the processes of collecting, classifying, storing information, etc.
Among the methods intended for assessing mentoring at the enterprise, we recommend using a system of evaluation indicators that will ensure quantitative assessment, reliability (objectivity), identify the level of achievement of goals and reserves of mentoring activities and develop directions for their application, etc. It should be noted that the optimal number of indicators is 5-25 pcs., they should all be quantitatively measured (in particular, as a result of expert assessments) and criterion (the obtained value of the indicator should indicate the level of achievement of the corresponding mentoring goal). We recommend a system of indicators covering 2 areas of evaluation: the level of mentoring processes in the enterprise, the effectiveness of the company's mentoring (Table 1).
number of employees involved in the mentoring process ( ) to planned ( ) Indicates the ratio of actual costs made to mentoring and other types of training in the enterprise ( ) to planned, which should be
fund ( ) employees ( ) Calculation 4 = = = − № 1 1 2 3
( )
=
− =
= = =
Expert evaluation according to the specified
criteria = Evaluation of the effectiveness of mentoring in the enterprise [0; 0,55] – managers are completely dissatisfied with the program; [0,56; 0,75] – managers are partially
satisfied with the mentoring program; [0,76; 1] – managers are loyal to the mentoring
program [0; 0,55] – low stability of the company's personnel;
[0,56; 0,75] – average stability of the company's personnel; [0,76; 1] –
stable staff [0; 0,55] – low level of professional development of employees; [0,56; 0,75] –
average level of professional development of employees; [0,76; 1] – high level of professional development of employees [0; 0,55] – low level; [0,56; 0,75] – average level; [0,76;
1] – high level of involvement of employees
in innovation activities [0; 0,55] – low level; [0,56; 0,75] – average level; [0,76; 1] – high level of staffing of the state of the enterprise 0 – employee's goals do not meet the goals of the enterprise; 0,5 – partial compliance with the goals of the employee and the enterprise; 1 – employee's goals fully meet the goals of [0; 0,55] – the level of increase in production is minimal; [0,56; 0,75] – average level; [0,76; 1] – the level of increase in production is optimal formula: ∑1=01 = 1.
We will evaluate mentoring activities at Lviv IT cluster enterprises. Lviv IT Cluster is a community of IT companies, government and education that have taken the responsibility to work on the development of the industry and the region through education, industry promo (city, region and companies) and infrastructure [36]. Cluster participants are companies of different sizes and with different needs, most of them have experience in mentoring activities that should be evaluated. Thus, the presence of different enterprises in the cluster allows you to evaluate individually by enterprises and identify the level of involvement of employees in mentoring activities in the cluster. To do this, we calculate the indicators intended for assessing mentoring at the enterprises of the cluster (Table 2).
= × + ×
+ + × + × + × × + + × + ×
+ × + × , (1) where – weight of indicators intended for the evaluation of mentoring activities in enterprises.
According to the recommendations provided in the economic literature on identifying the level of involvement of individuals in the learning process, the results obtained should be checked for statistical effects that can be evaluated using two-factual dispersion analysis of all components of involvement in mentoring processes [12]. To identify the difference in the level of involvement of various structural units of the enterprise (departments) in the mentoring processes. In order to identify differences in the statistically significant studied criteria of the groups (departments) and posterior comparisons should be made according to the Tukey criterion [31-35]. As a result, we will find out which departments (enterprises) differ in the level of implementation of individual mentoring components that characterize the recommended indicators (Table 1).
The information base for calculating indicators is the data of analytical and management accounting of enterprises, as well as expert assessments. On their basis, indicators should be calculated, as well as generalizations should be carried out (for example, determining the average level as provided for in calculating the average rank of compliance with goals, etc.). In order to summarize the results of the assessment, we propose to identify the integral level of indicators ( taking into account their weight. This is ensured in the process of using the factorial method by the ),
) )
)
) 10
( )
Evaluation of the effectiveness of mentoring in the enterprise
Lviv IT Cluster mission: "In the future, we see Lviv as a world-class high technology center. And our mission is to contribute to this as much as possible." The cluster includes companies, educational institutions, local authorities and BPO participants. According to this we will carry out an expert assessment of the weight of each of the indicators included in the recommended system (Table 1). This will reveal the level of mentoring activity at each of the enterprises of the Lviv IT cluster. The results are summarized in Table. 3.
Table 3 Weight of indicators intended for evaluation of mentoring activities at Lviv IT cluster enterprises
)
Weight 0,13 0,03 0,16 0,05 0,13 0,03 0,16 0,05 0,1 0,12
We base the obtained values of meaning of indicators (Table 2) and weight for enterprises of Lviv IT cluster (Table 3) in the equation (1). As a result, the level of mentoring activity ( ) of each of the enterprises was obtained: for the enterprise 1 =0,74; for the enterprise 2 =0,76; for the enterprise 3 =0,67; for the enterprise 4 =0,79.
Next, we will compare the levels of mentoring of enterprises in the Lviv IT cluster using the online service (https://astatsa.com/OneWay_Anova_with_TukeyHSD/), which conducts posterior comparisons according to the Tukey criterion. The results are summarized in Table. 4. Table 4
As a result of the calculations, we can affirm insignificant differences in the levels of mentoring for enterprises of the Lviv IT Cluster and a sufficient level of mentoring activities of the cluster. This is also evidenced by the data obtained as a result of the aggregation of indicators and the determination of the integral level of mentoring activities.
The article substantiated the importance of assessing mentoring as a method of personalized learning, which has a significant motivational impact on the employees of the enterprise and the effectiveness of their activities. To ensure this, approaches to the evaluation of mentoring at enterprises have been studied and summarized, as a result of which the procedure for evaluating mentoring activities and the scorecard are recommended. The recommended procedure contains the stages of mentoring assessment, and the indicators are grouped into two areas of evaluation: assessment of the level of mentoring processes in the enterprise (level of implementation of the mentoring plan, the level of costs for mentoring activities and other types of training of employees, the index of loyalty of employees to mentoring, the index of loyalty of managers to mentoring employees), determining the effectiveness of mentoring the enterprise (level of stability of the company's personnel, the level of professional development of employees, the level of involvement of personnel in innovation, the level of quantitative complexity of the states of the enterprise, the average rank of compliance with goals, the level of increase in productivity (production) in the enterprise). The recommended procedure provides for summarizing indicators and calculating the integral level of mentoring activities and identifying differences in the level of involvement in the mentoring processes of various structural units of the enterprise (departments), for which a posterior comparisons are provided according to the Tukey criterion. Practical approbation of the order took place on the basis of the Lviv IT Cluster, where there are minor differences in the levels of mentoring at enterprises.
Further research should explore the possibility of combining the developed order of mentoring evaluation with intelligent HR systems in the enterprise (DeloPro, SAP R/3, Baan, Oracle Applications, etc.).
[9] A. Mouzoune, Contribution to Enterprise Intelligent Systems Architecture: Assumptions, Expectations and a Proposal, Intelligent Information Management, Vollume 4, 6 (2012) 1-8.
DOI:10.4236/iim.2012.46035. [10] N. Shpak, K. Doroshkevych, Y. Shpak, I. Salata, M. Sharko, Strategy and Tactics of International Digitalization and Intellectualization of Economic Relations, CEUR Workshop Proceedings: Proceedings of 5th International Conference on Computational Linguistics and Intelligent Systems (COLINS-2021), Volume 2870 (2021) 1477-1487. URL: http://ceurws.org/Vol-2870/paper109.pdf. [11] A. J. Hobson, B. Maxwell, K. Káplár-Kodácsy, The Role of the Mentoring, Programme Coordinator: A Rapid Evidence Review Education & Training Foundation, University of Brighton and Sheffield Hallam University, (05) 2021. URL: https://www.etfoundation.co.uk/wpcontent/uploads/2021/05/ETF_MCoord_Report_UoB_SHU_29_April_2021_Final.pdf. [12] D. I. Martsenkovskyi, Vikovyi polimorfizm posttravmatychnykh stresovykh rozladiv ta depresii u ditei vnutrishno peremishchenykh osib z okupovanykh terytorii [Age polymorphism of posttraumatic stress disorders and depression in children of internally displaced persons from the occupied territories], Arkhiv psykhiatrii [Archives of Psychiatry], Volume 24, 1 (2018) 15-20.
URL: http://nbuv.gov.ua/UJRN/apsuh_2018_24_1_4. [13] J. Batut, M. Kvaskoff, M. C. Morris, When mentoring matters: a French mentoring program for women in science, Nature Biotechnology, 39 (2021) 776–779. URL: https://www.nature.com/articles/s41587-021-00951-2. [14] A. G. Opsahl, C. Townsend, Mentoring to engage diverse undergraduate nursing students in honors research, Nurs Forum, 56:19 (2021). DOI: https://doi.org/10.1111/nuf.12503. [15] D. Games, The Effectiveness of Mentoring and Coaching in Teaching Business Planning: The Case of Universitas Andalas, Advances in Social Science, Education and Humanities Research, Proceedings of the 3rd International Conference on Educational Development and Quality Assurance (ICED-QA 2020), 506 (2020) 1-5. [16] S. Resnjanskij, J. Ruhose, S. Wiederhold, L. Woessmann, Mentoring Improves the Labor-Market Prospects of Disadvantaged Adolescents, CESifo Forum, Volume 22, 4 (2021) 38-73. URL: https://www.cesifo.org/DocDL/CESifo-forum-2021-4-resnjanskij-etal-Mentoring-Improves-theLabor-Market-Prospects-of-Disadvantaged-Adolescents.pdf. [17] G. I. Egorova, Tehnologicheskij diskurs povyshenija urovnja professionalnyh kompetencij pedagoga [Technological discourse of increasing the level of professional competencies of a teacher], Humanities researches of the Central Russia, 1 (18) (2021) 45-63. URL: https://cyberleninka.ru/article/n/tehnologicheskiy-diskurs-povysheniya-urovnya-professionalnyhkompetentsiy-pedagoga/viewer. [18] S. Daniali, F. M. Khortabi, Y. Mohammadbeiki, Innovative approach for evaluating training effectiveness, Proceedings of the 32nd International Business Information Management Association Conference, IBIMA 2018 – Vision 2020: Sustainable Economic Development and Application of Innovation Management from Regional expansion to Global Growth, 32 (2018) 818-830. [19] K. K. Ilyushnikov, S. V. Lobova, Evaluation model for effectiveness of corporate training based on key performance indicators and detailed metrics, Espacios, Volume 40, 1 (2019), 29-37.
URL:http://www.revistaespacios.com/ a19v40n01/a19v40n01p29.pdf [20] E. N. Vetluzhskih, Obuchaem – ocenivaem. Modeli kriterialnoj ocenki rezultatov obuchenija [Models for Criteria-Based Assessment of Learning Outcomes], Spravochnik po upravleniju personalom [Human Resources Handbook], 2 (2005) 10-18. [21] T. L. Bezrukova et al, Optimization model of motivational process as a basis of management decisions, IOP Conf. Ser.: Mater. Sci. Eng, 483 012025 (2019), DOI:10.1088/1757899X/483/1/012025. [22] E. A. Belalova, Metody analiza jeffektivnosti sistemy razvitija personala organizacii [Methods for analyzing the effectiveness of the organization's personnel development system], Ekonomika i menedzhment innovacionnyh tehnologij [Economics and management of innovative technologies], 11 (2017). URL: https://ekonomika.snauka.ru/2017/11/15439