=Paper=
{{Paper
|id=Vol-2298/paper11
|storemode=property
|title=Development of Decision Making Software Agent for Efficiency Indicators System of IT-Specialists
|pdfUrl=https://ceur-ws.org/Vol-2298/paper11.pdf
|volume=Vol-2298
|authors=Ekaterina Tabatchikova,Konstantin Aksyonov,Igor Kalinin,Yuriy Chiryshev,Egor Talantsev,Andrey Tarasiev,Valeriy Kanev
}}
==Development of Decision Making Software Agent for Efficiency Indicators System of IT-Specialists==
https://ceur-ws.org/Vol-2298/paper11.pdf
Development of Decision Making Software Agent for Efficiency
Indicators System of IT-Specialists
Konstantin A. Aksyonov Igor A. Kalinin Ekaterina A. Tabatchikova
Ural Federal University LLC "UralInnovation" Ural Federal University
Ekaterinburg, Russia Ekaterinburg, Russia Ekaterinburg, Russia
bpsim.dss@gmail.com igor_kalinin@hotmail.com katuushik@gmail.com
Uriy V. Chiryshev Andrey Tarasiev Egor N. Talancev
Ural Federal University Ural Federal University Ural Federal University
Ekaterinburg, Russia Ekaterinburg, Russia Ekaterinburg, Russia
iurii.chiryshev@mail.ru andrew4800@mail.ru i.spyric@gmail.com
Valeriy S. Kanev
Siberian State University of
Telecommunications and
Informatics Novosibirsk, Russia
kanev@sibsutis.ru
Abstract
The article shows an example of the implementation of the basic part
of the software agent for the task of assessing the performance of IT
professionals. The technology of the software agent is developed with the
help of the voice and chat constructor TWIN.NLP and is oriented to work
with the voice interface and the processing of textual information (via the
chat-bot interface). The work defines the main concepts (entities) and
intentions for the task of assessing the performance indicators of IT
professionals.
Keywords: decision making, natural language processing, multi agent
technology.
1. Introduction
Many companies face the problem of assessing the work of IT professionals. In this regard, a number of questions arises:
what determines the effectiveness of their work, do IT professionals need the key performance indicators (KPI) system,
from which to build when creating it and what is the difference between KPI for developers? And, finally, can KPI
demotivate? KPI for IT department should be developed taking into account some features of this field of activity. After all,
IT-specialists are a specific category of employees.
To determine the results of work in companies, an estimate is applied based on the KPI indicators. The term KPI refers to a
set of key performance indicators of a company or its separate structural unit that contributes to achieving the strategic and
tactical goals. The use of KPI indicators provides companies with a tool to assess their current status and determines the
degree of compliance with the implementation of a previously defined development strategy.
Conventionally, in practice, the scheme of applying KPI is as follows: for the employee goals are set, according to the
results of which he receives an appropriate reward (bonus, intangible reward, etc.). Otherwise, there is no such
remuneration. This system allows you to make an assessment of the work of employees as efficiently as possible.
A key condition in the application of KPI is the compliance of the objectives with the following criteria [1]:
• Concreteness;
• Measurability;
• Reachability;
• Relevance;
• Limited in time.
In general, the KPI indicators of the IT department can be reduced to the following requirements:
• Degree of satisfaction of the internal or external customer;
• Compliance with the deadlines for the performance of work in accordance with the assigned terms of reference,
SLA, etc;
• The quality of the work performed;
• Reductions in costs and risks;
• Minimize the number of errors.
In the application of the KPI indicators, it is important not only to set goals and verify their implementation based on
performance. The key attention should be given to the analysis of the results, the reasons for not achieving any goals,
the revision of the approach to the definition of goals in the future. In addition, we must not forget that there must be
a connection between the fulfillment of the IT goals and the strategic goals of the whole company. In this case, the
KPI indicator system will be most effective and effective [2].
2. Studying the methodology of employee management based on KPI
The KPI system refers to the so-called meritocratic methods, that is, to approaches based on the evaluation of work
on real achievements using the mechanisms of objective measurements.
The KPI evaluation methodology assumes that for each position in the company, two models of current results and
competences have been developed (in the form of a table). The first lists all the criteria by which the effectiveness of
the employee should be assessed - quantitative and qualitative, team and individual. In the second - the competencies
required for this post - corporate, managerial and expert. From two models choose five-seven basic indicators for an
estimation of results of the competence of the employee for the concrete period and write down them in the table of
its personal activity. Competences at the same time are equal to the qualitative results of his work. The immediate
head of the employee assigns to each of the chosen indicators a weight from 0 to 1. At the same time he is guided only
by his own priorities. The total weight of indicators should be equal to one.
For all indicators, three levels of efficiency are initially set [3]:
• The base - is the starting point from which the result is counted. The worst value.
• A norm - is a level that must necessarily be achieved in the light of all the circumstances.
• The goal - is the level to be pursued, a kind of ideal indicator.
At the end of the control period, all KPI indicators are evaluated. At the same time, qualitative ones are estimated by
the order of 100-point scale, and quantitative ones by the natural metric scale. After the evaluation of the actual
indicator, the employee's personal result is calculated by the formula (1):
(Fact minus base / norm minus base) × 100% = result (%) (1)
The result reflects the degree of fulfillment or over fulfillment of the norm. After evaluating each indicator, the
employee's rating is evaluated, for this, the partial results are multiplied by the weight of the corresponding KPIs and
added up. As a result, they receive an average employee performance ratio. If it is more than 100% - this indicates
high performance, and if less - for example, that for some indicators, the employee has not achieved even a standard,
and the overall result of his work is lower than the established level.
The main advantage of the KPI system is that the company has a mechanism for evaluating their work that is
transparent for employees, and the management has a tool for correcting the work of subordinates in case the current
performance results are lagging behind the planned ones.
2.1 How should the KPI system be applied in practice?
Any indicator, for example, the degree of customer satisfaction, should have a weighted standard that can be gradually
changed from "loyal" to "tight", thus increasing the performance requirements. First of all, it is worthwhile to track
the dynamics of the selected indicators for at least a quarter, determine the standards and see how much they work. It
is believed that the IT service number of indicators should not be more than seven. And if the IT service consists of
several departments, then each department will have smaller indicators (2-3). If the values of the indicators constantly
exceed the norm, it will be necessary to understand the adequacy of the standard and the correctness of the
measurements. It is important that measurements are carried out with the help of certain tools or known facts. If the
figures are too low, this may indicate that the standard is too tight.
The correct organization of monitoring allows us to work out the standards of indicators for further implementation
for an annual perspective. After that they can already be introduced and tied to the system of motivation, namely to
the variable component of wages. The IT division refers to the service, the bonus for which is usually at the end of the
year, half a year or quarter. A more or less common option for supporting units is a variable component of about 20%
of annual revenue. But everything depends on the system of motivation and remuneration that is accepted in the
company. Very often, a project motivation system is applied, when the IT department is rewarded at the end of the
project: the implementation of the system, the completion of its configuration. The "motivational" budget must
necessarily be at the staff service and, accordingly, the head of the IT service, so that it was possible to encourage
those who were day and night at work [4].
For decision support system development used big data technology [5-6, 8, 13], multi agent approach [7, 9, 12-14]
and simulation [10-11].
3 Realization
To automatically determine the KPI IT specialist,was modeled a test agent for identifying entities and
intentions.Modeling - the study of objects of cognition on their models; the construction and study of models of really
existing objects, processes or phenomena in order to obtain explanations for these phenomena, as well as for predicting
the phenomena of interest to the researcher [7]. A named entity is a word or phrase intended for a specific, well-
defined object or phenomenon that distinguishes this object or phenomenon from a series of similar objects or
phenomena.The test agent is an automated system.Automated system - a collection of a managed object and automatic
control devices, in which part of the management functions are performed by a person [10].
The technology of the software agent is developed with the help of the voice and chat constructor TWIN.NLP and is
oriented to work with the voice interface and the processing of textual information (via the chat-bot interface).The
initial data can be seen in Figure 1:
Figure 1 – Initial data
Intention can be defined as the meaning of what has been said, that is, what the user meant by pronouncing a certain
phrase. The intentions for this subject area are shown in Figure 2:
Figure 2 - Making intentions
At the stage of the experiment in this program, you can set various queries, including the calculation of KPI, the
answers to which the program will give to the user.Decision making is the process of identifying alternatives and
choosing among them, based on the values and preferences of the decision maker [8]. The search result after learning
on the neural network can be seen in Figure 3:
Figure 3 - Result of the experiment
4. Conclusion
Work on the KPI system has many advantages:
• It is established that in companies with such a system employees work 20-30% more efficiently.
• Specialists will clearly understand which tasks are the most important and how to implement them.
• With a well-implemented system of indicators, the control over the company's activities is made much easier, due to
which problems are detected at the stage of their emergence and are resolved without harm.
• In the calculation of wages, the principle of justice operates: one who has worked diligently receives more. This
allows the organization to retain valuable personnel.
• The wage fund becomes a means of motivating staff, and not the main source of expenditure.
There are disadvantages of the KPI system. First of all, the downside is that it takes a lot of time and effort to
implement, because all the indicators need to be worked out in detail. Most likely, it will be necessary to retrain
employees, explain to them information about changes in working conditions and new tasks.
However, the main drawback is that efficiency is not always evaluated correctly. This can be avoided if, at the stage
of developing the system, it is impeccable to formulate criteria for the evaluation of the KPI.
The KPI system in the IT industry is an indicator of a certain level of maturity of the company and its business
processes. The goals that can be set for the KPI are diverse: this is an increase in the efficiency of the unit and each
employee, and the concentration of the IT resources on priority tasks for the company, and the introduction of a
transparent personnel assessment system, and optimization of the payroll.
It should be remembered that the introduction of the KPI system is not a panacea and does not replace the alignment
of business processes, nor it can be the only tool to motivate employees. But a competently developed KPI system
ensures the selection and retention of personnel focused on stable, reliable, focused and responsible work with
priorities for precise planning and compliance with deadlines and quality requirements.
5. Acknowledgment
The work was supported by Act 211 Government of the Russian Federation, contract № 02.A03.21.0006.
References
1.Gamblin, H. (2018). 6 Strategic KPIs for Your IT Department. Jive Communications.
2. IT KPI Examples: How Do You Measure IT Project Performance? (2018). Полученоиз Intelligent Business
Perfomance: https://www.bernardmarr.com/default.asp?contentID=1337
3. KPI Examples. (2018). Полученоиз Klipfolio: https://www.klipfolio.com/resources/kpi-examples
4. Scoreboard KPI Performance Management Software. (2018). Полученоиз Example KPIs for Information
Technology (IT) : https://kpidashboards.com/kpi/department/information-technology/
5. A. Borodin, S. Mirvoda, I. Kulikov, and S. Porshnev, “Optimization of Memory Operations in Generalized Search
Trees of PostgreSQL,” in Proceedings of International Conference: Beyond Databases, Architectures and Structures,
2017, pp. 224–232.
6. A. Borodin, S. Mirvoda, S. Porshnev, and M. Bakhterev, “Improving penalty function of R-tree over generalized
index search tree possible way to advance performance of PostgreSQL cube extension,” in Proceedings of IEEE 2nd
International Conference on Big Data Analysis (ICBDA), 2017, pp. 130–133.
7. B. Klebanov, T. Antropov, E. Riabkina, “The principles of multi-agent models of development based on the needs
of the agents,” in Proceedings of 35th Chinese Control Conference (CCC), 2016, pp. 7551–7555.
8. V. V. Alexandrov, S. V. Kuleshov, A. A. Zaytseva, “Active Data in Digital Software Defined Systems Based on
SEMS Structures”, Logical Analysis of Data and Knowledge with Uncertainties in SEMS – Smart Electromechanical
Systems, Studies in Systems, Decision and Control, pp. 61–69, 2016.
9. E. B. Yudin, M. N. Yudina, “Calculation of number of motifs on three nodes using random sampling of frames in
networks with directed links”, 2017 Siberian Symposium on Data Science and Engineering (SSDSE), pp. 23–26,
2017.
10. Sokolov B.V., Pavlov A.N., Yusupov R.M., Ohtilev M.U., Potryasaev S.A. Theoretical and technological
foundations of complex objects proactive monitoring management and control // Proceedings of the Symposium
Automated Systems and Technologies Peter the Great St. Petersburg Polytechnic University, Leibniz Universität
Hannover. 2015. P. 103-110.
11. Solovyeva I., Sokolov B., Ivanov D. Analysis of position optimization method applicability in supply chain
management problem // 2015 International Conference "Stability and Control Processes" in Memory of V.I. Zubov
(SCP) 2015. P. 498-500.
12. Klebanov B., Antropov, T., Riabkina, E. Bases of imitation model of artificial society construction accounting of
the agents’ needs recursion. 16th International Multidisciplinary Scientific GeoConference SGEM 2016.Book2
Vol. 1, 101-108 pp. DOI: 10.5593/SGEM2016/B21/S07.014, WOS:000395499400014.
13. Aksyonov K., Antonova A. Application of a metallurgical enterprise information system for collection and
analysis of big data and optimization of multi-agent resource conversion processes. 2018 International Research
Workshop on Information Technologies and Mathematical Modeling for Efficient Development of Arctic Zone, IT
and MathAZ 2018; Graduate School of Business and Management. Yekaterinburg; Russian Federation; 19-21 April
2018; Code 137083. CEUR Workshop Proceedings. Volume 2109, 2018, Pages 1-6.
14. Aksyonov K., Bykov E., Aksyonova O., Goncharova N., Nevolina A. Multi-agent simulation of the resource
conversion processes with the sample application to the logistical problem. UKSim-AMSS 11th European Modelling
Symposium on Mathematical Modelling and Computer Simulation. Manchester, England, 20 - 22 November 2017.
Pages177-181. DOI 10.1109/EMS.2017.39