Electronic training polygon for artificial intelligence systems A.A. Karandeev1,2, V.I. Baluta1,2, V.P. Osipov1 karalex755@gmail.com | vbaluta@keldysh.ru | osipov@keldysh.ru 1 Keldysh Institute of Applied Mathematics, Russian Academy of Sciences, Moscow, Russia 2 Plekhanov Russian University of Economics In the framework of the concept of neoconflictology, the possibilities and methods of mathematical modelling of conflict dynamics under uncertainty are considered. Complex contradictory situations, when it is necessary to quickly respond to changes in the situation and perform some actions in conditions of uncertainty, are often found in different spheres of activity. In this case, the uncertainty may be due to incomplete knowledge of the situation, the inability to quickly understand and evaluate possible options for its development, the influence on it from other participants in the process, etc. The result of further developments depends on how the decision will be to the situation and what actions will follow on its basis. Usually, the effectiveness of behavior in difficult situations is determined by the level of special training and experience of the decision maker. The trend of using artificial intelligence systems to support decision- making, including complex conflict situations with a high level of uncertainty, is now more frequent. Focusing on the use of such systems requires the creation and improvement of specialized tools and technologies, thanks to which the artificial intelligence systems themselves can form an information base, which is a necessary factor for developing rational solutions in a complex environment. The creation of a virtual polygons is one of the ways for producing such information base. Keywords: neoconflictology, conflict, training polygon, intelligent agent, Artificial intelligence, modelling. other agents in their phase space. The purpose of these 1. Introduction repeated experiments is to train the agent for rational In an antagonistic conflict, the effectiveness of action in the given conditions. To achieve this effect, a achieving goals depends on the adopted strategy of mathematical model is designed. behaviour, the speed of decision-making in a changing 2. Basic provisions of the model approach environment and the level of their optimality, or in other words, the degree of their compliance with the current The developed research technology is based on situation. numerical modelling of possible ways to actualize an The best way to analyze situations of this kind that antagonistic conflict based on the following demonstrate emerging phenomena or generate unforeseen considerations. In the simulation model, each of the patterns is to model and simulate them [1]. The subjects of the conflict can be represented as an problematic complexity of solving these issues is directly “intelligent” agent. The theory of agents or multi-agent related to the level of uncertainty, which is often due to a systems [4] is a computer theory which seeks to apprehend lack of time to obtain the information necessary for the coordination of competing independent process. An decision-making in a conflict situation and many other agent is thus a computer process [5], which can be factors. considered as autonomous since it is capable of adapting Given the modern development of computer methods when its environment changes. The basic characteristics of and tools, it is numerical modelling that assumes the role such an agent are the formation of an internal model of the of the tool by which you can explore a variety of conflicts surrounding world and the presentation of its place in it, a to identify the role and significance of certain behavioural system of rules for creating and changing this model, strategies, determine the list of effective actions in various decision-making methods for taking actions to respond situations. One of the methods that greatly simplify the adequately in response to a changing situation. Intelligent modelling process is the creation of an electronic polygon. agents have targeted behaviour to achieve a certain goal in Despite the complexity of developing such kind of the most optimal way. software systems, they make it possible to visually This process comes down to setting optimization simulate various situations and see what these or those problems and finding their solutions. Based on the solutions lead to. described representations, each of the subjects of the The main participants in a computer polygon are confrontation can be represented in the form of a complex intelligent agents [2-3] who are trying to achieve their system that has certain resources and has its own goals. It goals. They can have both common goals and completely is worth considering that many types of conflicts, such as opposite. It is worth considering that for each of the social and political, cannot be strictly defined [6]. Because participants in the conflict, their own space of probable of this, it may be necessary to discard some non-significant states can be formed, due to its capabilities and ideas about parameters. the "world in which it operates." Moreover, the state of the The subject's capabilities depend on his position in the agent can be changed not only due to the actions taken by phase state space at each current point in time, which is him, but also under the influence of other agents who are characterized by a certain set of particular characteristics. most often the adversaries. The agent’s task is to construct Otherwise, a change in its state over time depends on the an optimal trajectory for transition from its initial state to position of the subject in the phase space of his states, a given target state. which also include the physical coordinates of his location. Usually, the set of actions that an agent can perform is Moreover, each subject has its own phase space, only limited, but each of these actions must be taken into partial intersection with the opponent’s space is possible. account not only affects its own state, but also the state of The transition of the subject from the initial state to the Copyright © 2020 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0) target is carried out through a set of intermediate states, the state to another involves a certain sequence of actions trajectory of motion in phase space is a result of their (steps), which is determined by the subject's ideas about change. the cost of resources for each of them and an assessment Conventionally, the reflection of the model of the of their proximity to the target state. The diagram shows intelligent agent in the form of a graph, according to [7- that each individual action is associated with a certain 10], can be displayed in the form of the circuit shown in resource of the real world, the availability of which (in the Fig. 1. The initial position is conditionally shown in a blue representation of the object) is a factor determining the circle, and the target in yellow. Choosing a path from one possibility of its implementation. Fig. 1. Intelligent agent action planning In other words, any change in the parameter that initial position of the subject. We also set the second point describes some of the characteristics of the subject’s denoting the position of the target. The state of the subject position in the phase space can be associated with a change changes as a result of some elementary actions in the state in some scalar quantity — the transition price, which is space, while the set of possible actions is limited and is understood as the conditional cost of the action. Each of described by a set of rules of behavior, each of which is the parameters has its own “price scale”, which displayed by a vector in the phase space. The sequential dynamically depends on the current state of the subject and step-by-step execution of actions forms a certain trajectory is determined by the value of this parameter to solve the of the subject in the state space. In a two-dimensional problem. interpretation, the task is shown schematically in Fig. 2. 3. Adapting rules of conduct In the phase space or in the space of states we define some arbitrary point, which will be considered as the Fig. 2. Schematic representation of the problem The shortest direction or trajectory from the source to action vector, is unlikely to coincide with the shortest the target state is shown by a dashed line. It is obvious that direction. Clearly, a rational trajectory of movement the result of the implementation of one or another rule of should be as close to this line as possible. behavior, which is displayed in the phase space by some But in conditions of inaccurate information about the an operation can be done “manually”, based on visual properties of each of the vectors, it is difficult to construct. estimates. One of the possible results may be, for example, For the two-dimensional problem shown in figure 2, such the path options shown in Fig. 3. Fig. 3. A schematic example of the construction of rational trajectories In the presented schematic example, based on a visual by the degree of approximation of the new state to the assessment of the set of action vectors, such trajectories target state obtained as a result of this action from this can be constructed “manually”. There are at least two position. With a positive effect (approaching the goal), rational options for the trajectories of the transition from incentive is introduced, with a negative effect (moving the initial state to the target, and it is enough to use a away from the goal) - a penalty. At the beginning of the combination of only two of all possible actions. When calculation, all ratings are set close to zero. comparing the obtained trajectories, it is seen that that the The next move step is randomly selected from the list upper one is slightly more preferable in terms of the degree of all possible actions, but taking into account their rating. of approximation to the target position, but its The probability of choosing an action increases in implementation requires one more step. To determine the proportion to the rating. As a result of the calculations, the best of them, additional evaluation criteria are needed, for initially uniform distribution of the rating of actions is example, a comparison of criteria for accuracy and gradually deformed towards growing ratings. At each resource costs. Based on the above examples, it can be iteration, the current state in which the subject was located argued that in a multidimensional phase space, the is remembered. Thus, a model of stereotypical situations construction of the optimal path is possible only with full is formed in which the effects of the same actions can knowledge of the situation: where, by what means, with manifest themselves in different ways. Each iteration stops what intermediate results certain actions arising from the after reaching the target state or after completing a given rules of behaviour of agents can be realized. number of steps. At each subsequent iteration, the ratings The following scheme was proposed and tested. accumulated in previous experiments are saved and used Multidimensional phase space and an unordered set of when choosing actions. Therefore, the knowledge base is permissible actions are set. At the same time, the number accumulated and the conditional agent is “trained” in the of actions and the dimension of space are determined by rational choice of actions depending on the position in the the conditions of a specific task. It is assumed that the basis state space in which it is located. for decision-making by the agent in the current situation to choose a specific action is some a priori assessment of the 4. Results value of certain actions. Initially, it can be set by experts Testing of the methodology was carried out on an or randomly generated. arbitrary example in a space of 12 measurements with a Further refinement of the assessment of the actions that total number of rules of behavior equal to 50 (Fig. 4). the agent must carry out is the result of a wide series of The fig. 4 shows 33 of 50 actions in a 12-dimensional numerical experiments, in each of which the trajectory of state space. The positions of the initial and target states motion of the agent states in the phase space from the were randomly selected as vectors in the phase space. In source to the target is constructed. In the process of addition, in the form of increments of vectors, action performing calculations, a certain rating is assigned to vectors were arbitrarily specified. each perfect action. The rating of each action is determined Fig. 4. List of available actions Let us dwell in more detail on the data given in the The data given in the table reflect the obtained picture table. As already mentioned above, the agent’s training of the learning outcomes at some intermediate stage of the technology is demonstrated using a hypothetical example, experiments. For clarity, a visual analysis introduced the in which a certain zone of the phase space is color of the cells. In this case, color means the directivity conventionally defined as a range of changes in the of the effect, and the intensity of the color is its immediate coordinate values of the parameters, and the initial and meaning. In other words, the cells highlighted in red target position of the agent are determined inside this zone. indicate that the choice of this action will lead to the loss Variants of the agent’s possible actions are randomly of a parameter or resource. Moreover, the more intense the generated by coordinatewise specifying a certain set of color, the greater the loss. Positive effects highlighted in increment vectors. When generating incremental vectors, green. the only condition is to limit the length of each of these Fig. 5 shows the result of a computational procedure vectors to a value significantly less than the distance for finding a rational trajectory. between the initial and target position of the agent in the phase space. Fig. 5. Schematic representation of the problem The process of qualitatively improving the procedure application-oriented simulation, prediction and for choosing rational actions from a given set is visible in expertizes of conflict interaction, 2018. the fig. 5. The x-axis shows the number of experiments, in [4] Bousquet F., Barreteau O., Le C., Mullon C., Weber each of which a motion pattern limited in the number of J., An environmental modelling approach. The use of steps is plotted in the phase space, and the y-axis is the multi-agent simulations, 1999. distance from the target state in the adopted metric. It can [5] Ferber J., La kénétique: dés systems multi-agents à be seen that the agent training process leads to the une science de l'interaction, 1994. achievement of some non-improved level of performance, [6] Petukhov A., Мalhanov A., Sandalov V., Petukhov which, however, allows to get close enough to the target V., Mathematical Modeling of the Ethno-social state. Conflicts by Non-linear Dynamics. Proceedings of the 7th International Conference on Simulation and 5. Conclusions Modeling Methodologies, Technologies and Proposed technological solution for modeling the Applications, 2017. behavior of intelligent agents provides a fairly broad basis [7] A.Kh. Khakimova, O.V. Zolotarev, M.A. Berberova. for studying the features of constructing trajectories of Visualization of bibliometric networks of scientific achieving goals in phase space and the adaptive behavior publications on the study of the human factor in the of agents in a conflict situation. operation of nuclear power plants based on the Due to the specifics of the problem, approaches and bibliographic database Dimensions. Scientific methods for modeling the phase space of states of a Visualization, 2020, volume 12, number 2, pages 127- conflict environment are proposed, which allow 138, DOI: 10.26583/sv.12.2.10, E-ISSN:2079-3537. determining strategies for rational trajectories of goal [8] M.A.Berberova, K.I.Chernyavskii, «Comparative achievement. The technology for modeling agent behavior assessment of the NPP risk (on the example of Rostov is based on a detailed description of their behavior based and Kalinin NPP). Development of risk indicators on a scheme of intelligent transitions between states and atlas for Russian NPPs», GraphiCon 2019 Computer behavior models. Graphics and Vision. The 29th International Such a detailed description of complex behavior Conference on Computer Graphics and Vision. models makes it possible to uniformly display in the model Conference Proceedings (2019), Bryansk, Russia, of a conflict environment the essential aspects of the September 23-26, 2019, Vol-2485, urn:nbn:de:0074- behavior of real-world prototype objects. The proposed 2485-1, ISSN 1613-0073, DOI: 10.30987/graphicon- scheme for ensuring the adaptive behavior of agents in a 2019-2-290-294, http://ceur-ws.org/Vol- conflict environment is an integral part of the 2485/paper67.pdf, p. 290-294. methodological and algorithmic support of an electronic [9] M.A.Berberova, A.S.Oboimov, A.Kh.Khakimova, training ground for studying the conflict interaction of O.V.Zolotarev, «Risk-informed security system. The complex systems. use of surveillance cameras for the particularly According to the authors, this technology has shown its hazardous facilities safety», GraphiCon 2019 effectiveness in test tasks and can be recommended for use Computer Graphics and Vision. The 29th International as part of an electronic training polygon to test its Conference on Computer Graphics and Vision. capabilities in developing strategies for agents' behavior Conference Proceedings (2019), Bryansk, Russia, under conditions of uncertainty during their interaction. September 23-26, 2019, Vol-2485, urn:nbn:de:0074- A similar technology can be applied in automated 2485-1, ISSN 1613-0073, DOI: 10.30987/graphicon- process control support systems, where an operational 2019-2-321-325, http://ceur-ws.org/Vol- assessment of the situation is necessary based on the 2485/paper74.pdf, p. 321-325. available incomplete or inaccurate data with the [10] Baluta V., Osipov V., Chetverushkin B., Yakovenko development of recommendations for rational actions. In Y., Conceptual issues of model representation of particular, when creating combat control systems, conflicts. Proceedings of the International Scientific ensuring the safety of facilities, responding in emergency Conference, 2019. situations, as well as in control systems for unmanned About the authors vehicles. Karandeev Alexander A., phd student and Junior Scientific References Associate of the Keldysh Institute of Applied Mathematics, Russian Academy of Sciences. E-mail: KarAlex755@gmail.com [1] Klügl F., Oechslein C., Puppe F., Dornhaus A., Multi- Baluta Viktor I., PhD in Technical Science, Docent, Senior Agent Modelling in Comparison to Standard Scientific Associate of the Keldysh Institute of Applied Modelling, 2014. Mathematics, Russian Academy of Sciences. E-mail: [2] Nechaev Y., Osipov V., Chetverushkin B., Baluta V., vbaluta@keldysh.ru Ontological synthesis of management decisions in Osipov Vladimir P., PhD in Technical Science, Docent, Lead conditions of antagonistic conflicts, 2018. Scientist of the Keldysh Institute of Applied Mathematics, [3] Baluta V., Nechaev Y., Osipov V., Chetverushkin B., Russian Academy of Sciences. E-mail: osipov@keldysh.ru Conceptual base of a supercomputer platform of