The article shows an example of the implementation of the basic part of the software agent for the task of assessing the performance of IT professionals. The technology of the software agent is developed with the help of the voice and chat constructor TWIN.NLP and is oriented to work with the voice interface and the processing of textual information (via the chat-bot interface). The work defines the main concepts (entities) and intentions for the task of assessing the performance indicators of IT professionals.
Introduction Many companies face the problem of assessing the work of IT professionals. In this regard, a number of questions arises: what determines the effectiveness of their work, do IT professionals need the key performance indicators (KPI) system, from which to build when creating it and what is the difference between KPI for developers? And, finally, can KPI demotivate? KPI for IT department should be developed taking into account some features of this field of activity. After all, IT-specialists are a specific category of employees.
To determine the results of work in companies, an estimate is applied based on the KPI indicators. The term KPI refers to a set of key performance indicators of a company or its separate structural unit that contributes to achieving the strategic and tactical goals. The use of KPI indicators provides companies with a tool to assess their current status and determines the degree of compliance with the implementation of a previously defined development strategy.
Conventionally, in practice, the scheme of applying KPI is as follows: for the employee goals are set, according to the
results of which he receives an appropriate reward (bonus, intangible reward, etc.). Otherwise, there is no such
remuneration. This system allows you to make an assessment of the work of employees as efficiently as possible.
A key condition in the application of KPI is the compliance of the objectives with the following criteria [
In general, the KPI indicators of the IT department can be reduced to the following requirements: • Degree of satisfaction of the internal or external customer; • Compliance with the deadlines for the performance of work in accordance with the assigned terms of reference, SLA, etc; • The quality of the work performed; • Reductions in costs and risks; • Minimize the number of errors.
In the application of the KPI indicators, it is important not only to set goals and verify their implementation based on
performance. The key attention should be given to the analysis of the results, the reasons for not achieving any goals,
the revision of the approach to the definition of goals in the future. In addition, we must not forget that there must be
a connection between the fulfillment of the IT goals and the strategic goals of the whole company. In this case, the
KPI indicator system will be most effective and effective [
The KPI system refers to the so-called meritocratic methods, that is, to approaches based on the evaluation of work on real achievements using the mechanisms of objective measurements.
The KPI evaluation methodology assumes that for each position in the company, two models of current results and competences have been developed (in the form of a table). The first lists all the criteria by which the effectiveness of the employee should be assessed - quantitative and qualitative, team and individual. In the second - the competencies required for this post - corporate, managerial and expert. From two models choose five-seven basic indicators for an estimation of results of the competence of the employee for the concrete period and write down them in the table of its personal activity. Competences at the same time are equal to the qualitative results of his work. The immediate head of the employee assigns to each of the chosen indicators a weight from 0 to 1. At the same time he is guided only by his own priorities. The total weight of indicators should be equal to one.
For all indicators, three levels of efficiency are initially set [3]: • The base - is the starting point from which the result is counted. The worst value. • A norm - is a level that must necessarily be achieved in the light of all the circumstances. • The goal - is the level to be pursued, a kind of ideal indicator.
At the end of the control period, all KPI indicators are evaluated. At the same time, qualitative ones are estimated by the order of 100-point scale, and quantitative ones by the natural metric scale. After the evaluation of the actual indicator, the employee's personal result is calculated by the formula (1):
(Fact minus base / norm minus base) × 100% = result (%) (1) The result reflects the degree of fulfillment or over fulfillment of the norm. After evaluating each indicator, the employee's rating is evaluated, for this, the partial results are multiplied by the weight of the corresponding KPIs and added up. As a result, they receive an average employee performance ratio. If it is more than 100% - this indicates high performance, and if less - for example, that for some indicators, the employee has not achieved even a standard, and the overall result of his work is lower than the established level.
The main advantage of the KPI system is that the company has a mechanism for evaluating their work that is transparent for employees, and the management has a tool for correcting the work of subordinates in case the current performance results are lagging behind the planned ones.
Any indicator, for example, the degree of customer satisfaction, should have a weighted standard that can be gradually changed from "loyal" to "tight", thus increasing the performance requirements. First of all, it is worthwhile to track the dynamics of the selected indicators for at least a quarter, determine the standards and see how much they work. It is believed that the IT service number of indicators should not be more than seven. And if the IT service consists of several departments, then each department will have smaller indicators (2-3). If the values of the indicators constantly exceed the norm, it will be necessary to understand the adequacy of the standard and the correctness of the measurements. It is important that measurements are carried out with the help of certain tools or known facts. If the figures are too low, this may indicate that the standard is too tight.
The correct organization of monitoring allows us to work out the standards of indicators for further implementation for an annual perspective. After that they can already be introduced and tied to the system of motivation, namely to the variable component of wages. The IT division refers to the service, the bonus for which is usually at the end of the year, half a year or quarter. A more or less common option for supporting units is a variable component of about 20% of annual revenue. But everything depends on the system of motivation and remuneration that is accepted in the company. Very often, a project motivation system is applied, when the IT department is rewarded at the end of the project: the implementation of the system, the completion of its configuration. The "motivational" budget must necessarily be at the staff service and, accordingly, the head of the IT service, so that it was possible to encourage those who were day and night at work [4].
For decision support system development used big data technology [
To automatically determine the KPI IT specialist,was modeled a test agent for identifying entities and
intentions.Modeling - the study of objects of cognition on their models; the construction and study of models of really
existing objects, processes or phenomena in order to obtain explanations for these phenomena, as well as for predicting
the phenomena of interest to the researcher [
The technology of the software agent is developed with the help of the voice and chat constructor TWIN.NLP and is oriented to work with the voice interface and the processing of textual information (via the chat-bot interface).The initial data can be seen in Figure 1:
Figure 1 – Initial data
Intention can be defined as the meaning of what has been said, that is, what the user meant by pronouncing a certain
phrase. The intentions for this subject area are shown in Figure 2:
At the stage of the experiment in this program, you can set various queries, including the calculation of KPI, the
answers to which the program will give to the user.Decision making is the process of identifying alternatives and
choosing among them, based on the values and preferences of the decision maker [
Work on the KPI system has many advantages: • It is established that in companies with such a system employees work 20-30% more efficiently. • Specialists will clearly understand which tasks are the most important and how to implement them. • With a well-implemented system of indicators, the control over the company's activities is made much easier, due to which problems are detected at the stage of their emergence and are resolved without harm. • In the calculation of wages, the principle of justice operates: one who has worked diligently receives more. This allows the organization to retain valuable personnel. • The wage fund becomes a means of motivating staff, and not the main source of expenditure. There are disadvantages of the KPI system. First of all, the downside is that it takes a lot of time and effort to implement, because all the indicators need to be worked out in detail. Most likely, it will be necessary to retrain employees, explain to them information about changes in working conditions and new tasks.
However, the main drawback is that efficiency is not always evaluated correctly. This can be avoided if, at the stage of developing the system, it is impeccable to formulate criteria for the evaluation of the KPI.
The KPI system in the IT industry is an indicator of a certain level of maturity of the company and its business processes. The goals that can be set for the KPI are diverse: this is an increase in the efficiency of the unit and each employee, and the concentration of the IT resources on priority tasks for the company, and the introduction of a transparent personnel assessment system, and optimization of the payroll.
It should be remembered that the introduction of the KPI system is not a panacea and does not replace the alignment of business processes, nor it can be the only tool to motivate employees. But a competently developed KPI system ensures the selection and retention of personnel focused on stable, reliable, focused and responsible work with priorities for precise planning and compliance with deadlines and quality requirements.
The work was supported by Act 211 Government of the Russian Federation, contract № 02.A03.21.0006.