In an antagonistic conflict, the effectiveness of achieving goals depends on the adopted strategy of behaviour, the speed of decision-making in a changing environment and the level of their optimality, or in other words, the degree of their compliance with the current situation.
The best way to analyze situations of this kind that demonstrate emerging phenomena or generate unforeseen patterns is to model and simulate them [1]. The problematic complexity of solving these issues is directly related to the level of uncertainty, which is often due to a lack of time to obtain the information necessary for decision-making in a conflict situation and many other factors.
Given the modern development of computer methods and tools, it is numerical modelling that assumes the role of the tool by which you can explore a variety of conflicts to identify the role and significance of certain behavioural strategies, determine the list of effective actions in various situations. One of the methods that greatly simplify the modelling process is the creation of an electronic polygon. Despite the complexity of developing such kind of software systems, they make it possible to visually simulate various situations and see what these or those solutions lead to.
The main participants in a computer polygon are intelligent agents [2][3] who are trying to achieve their goals. They can have both common goals and completely opposite. It is worth considering that for each of the participants in the conflict, their own space of probable states can be formed, due to its capabilities and ideas about the "world in which it operates." Moreover, the state of the agent can be changed not only due to the actions taken by him, but also under the influence of other agents who are most often the adversaries. The agent's task is to construct an optimal trajectory for transition from its initial state to a given target state.
Usually, the set of actions that an agent can perform is limited, but each of these actions must be taken into account not only affects its own state, but also the state of other agents in their phase space. The purpose of these repeated experiments is to train the agent for rational action in the given conditions. To achieve this effect, a mathematical model is designed.
The developed research technology is based on numerical modelling of possible ways to actualize an antagonistic conflict based on the following considerations. In the simulation model, each of the subjects of the conflict can be represented as an "intelligent" agent. The theory of agents or multi-agent systems [4] is a computer theory which seeks to apprehend the coordination of competing independent process. An agent is thus a computer process [5], which can be considered as autonomous since it is capable of adapting when its environment changes. The basic characteristics of such an agent are the formation of an internal model of the surrounding world and the presentation of its place in it, a system of rules for creating and changing this model, decision-making methods for taking actions to respond adequately in response to a changing situation. Intelligent agents have targeted behaviour to achieve a certain goal in the most optimal way. This process comes down to setting optimization problems and finding their solutions. Based on the described representations, each of the subjects of the confrontation can be represented in the form of a complex system that has certain resources and has its own goals. It is worth considering that many types of conflicts, such as social and political, cannot be strictly defined [6]. Because of this, it may be necessary to discard some non-significant parameters.
The subject's capabilities depend on his position in the phase state space at each current point in time, which is characterized by a certain set of particular characteristics. Otherwise, a change in its state over time depends on the position of the subject in the phase space of his states, which also include the physical coordinates of his location. Moreover, each subject has its own phase space, only partial intersection with the opponent's space is possible. The transition of the subject from the initial state to the target is carried out through a set of intermediate states, the trajectory of motion in phase space is a result of their change.
Conventionally, the reflection of the model of the intelligent agent in the form of a graph, according to [7][8][9][10], can be displayed in the form of the circuit shown in Fig. 1. The initial position is conditionally shown in a blue circle, and the target in yellow. Choosing a path from one state to another involves a certain sequence of actions (steps), which is determined by the subject's ideas about the cost of resources for each of them and an assessment of their proximity to the target state. The diagram shows that each individual action is associated with a certain resource of the real world, the availability of which (in the representation of the object) is a factor determining the possibility of its implementation.
In other words, any change in the parameter that describes some of the characteristics of the subject's position in the phase space can be associated with a change in some scalar quantity -the transition price, which is understood as the conditional cost of the action. Each of the parameters has its own "price scale", which dynamically depends on the current state of the subject and is determined by the value of this parameter to solve the problem.
In the phase space or in the space of states we define some arbitrary point, which will be considered as the initial position of the subject. We also set the second point denoting the position of the target. The state of the subject changes as a result of some elementary actions in the state space, while the set of possible actions is limited and is described by a set of rules of behavior, each of which is displayed by a vector in the phase space. The sequential step-by-step execution of actions forms a certain trajectory of the subject in the state space. In a two-dimensional interpretation, the task is shown schematically in Fig. 2.
The shortest direction or trajectory from the source to the target state is shown by a dashed line. It is obvious that the result of the implementation of one or another rule of behavior, which is displayed in the phase space by some action vector, is unlikely to coincide with the shortest direction. Clearly, a rational trajectory of movement should be as close to this line as possible.
But in conditions of inaccurate information about the properties of each of the vectors, it is difficult to construct. For the two-dimensional problem shown in figure 2, such an operation can be done "manually", based on visual estimates. One of the possible results may be, for example, the path options shown in Fig. 3.
In the presented schematic example, based on a visual assessment of the set of action vectors, such trajectories can be constructed "manually". There are at least two rational options for the trajectories of the transition from the initial state to the target, and it is enough to use a combination of only two of all possible actions. When comparing the obtained trajectories, it is seen that that the upper one is slightly more preferable in terms of the degree of approximation to the target position, but its implementation requires one more step. To determine the best of them, additional evaluation criteria are needed, for example, a comparison of criteria for accuracy and resource costs. Based on the above examples, it can be argued that in a multidimensional phase space, the construction of the optimal path is possible only with full knowledge of the situation: where, by what means, with what intermediate results certain actions arising from the rules of behaviour of agents can be realized.
The following scheme was proposed and tested. Multidimensional phase space and an unordered set of permissible actions are set. At the same time, the number of actions and the dimension of space are determined by the conditions of a specific task. It is assumed that the basis for decision-making by the agent in the current situation to choose a specific action is some a priori assessment of the value of certain actions. Initially, it can be set by experts or randomly generated.
Further refinement of the assessment of the actions that the agent must carry out is the result of a wide series of numerical experiments, in each of which the trajectory of motion of the agent states in the phase space from the source to the target is constructed. In the process of performing calculations, a certain rating is assigned to each perfect action. The rating of each action is determined by the degree of approximation of the new state to the target state obtained as a result of this action from this position. With a positive effect (approaching the goal), incentive is introduced, with a negative effect (moving away from the goal) -a penalty. At the beginning of the calculation, all ratings are set close to zero.
The next move step is randomly selected from the list of all possible actions, but taking into account their rating. The probability of choosing an action increases in proportion to the rating. As a result of the calculations, the initially uniform distribution of the rating of actions is gradually deformed towards growing ratings. At each iteration, the current state in which the subject was located is remembered. Thus, a model of stereotypical situations is formed in which the effects of the same actions can manifest themselves in different ways. Each iteration stops after reaching the target state or after completing a given number of steps. At each subsequent iteration, the ratings accumulated in previous experiments are saved and used when choosing actions. Therefore, the knowledge base is accumulated and the conditional agent is "trained" in the rational choice of actions depending on the position in the state space in which it is located.
Testing of the methodology was carried out on an arbitrary example in a space of 12 measurements with a total number of rules of behavior equal to 50 (Fig. 4).
The fig. 4 shows 33 of 50 actions in a 12-dimensional state space. The positions of the initial and target states were randomly selected as vectors in the phase space. In addition, in the form of increments of vectors, action vectors were arbitrarily specified.
Let us dwell in more detail on the data given in the table. As already mentioned above, the agent's training technology is demonstrated using a hypothetical example, in which a certain zone of the phase space is conventionally defined as a range of changes in the coordinate values of the parameters, and the initial and target position of the agent are determined inside this zone. Variants of the agent's possible actions are randomly generated by coordinatewise specifying a certain set of increment vectors. When generating incremental vectors, the only condition is to limit the length of each of these vectors to a value significantly less than the distance between the initial and target position of the agent in the phase space.
The data given in the table reflect the obtained picture of the learning outcomes at some intermediate stage of the experiments. For clarity, a visual analysis introduced the color of the cells. In this case, color means the directivity of the effect, and the intensity of the color is its immediate meaning. In other words, the cells highlighted in red indicate that the choice of this action will lead to the loss of a parameter or resource. Moreover, the more intense the color, the greater the loss. Positive effects highlighted in green.
Fig. 5 shows the result of a computational procedure for finding a rational trajectory.
The process of qualitatively improving the procedure for choosing rational actions from a given set is visible in the fig. 5. The x-axis shows the number of experiments, in each of which a motion pattern limited in the number of steps is plotted in the phase space, and the y-axis is the distance from the target state in the adopted metric. It can be seen that the agent training process leads to the achievement of some non-improved level of performance, which, however, allows to get close enough to the target state.
Proposed technological solution for modeling the behavior of intelligent agents provides a fairly broad basis for studying the features of constructing trajectories of achieving goals in phase space and the adaptive behavior of agents in a conflict situation.
Due to the specifics of the problem, approaches and methods for modeling the phase space of states of a conflict environment are proposed, which allow determining strategies for rational trajectories of goal achievement. The technology for modeling agent behavior is based on a detailed description of their behavior based on a scheme of intelligent transitions between states and behavior models. Such a detailed description of complex behavior models makes it possible to uniformly display in the model of a conflict environment the essential aspects of the behavior of real-world prototype objects. The proposed scheme for ensuring the adaptive behavior of agents in a conflict environment is an integral part of the methodological and algorithmic support of an electronic training ground for studying the conflict interaction of complex systems.
According to the authors, this technology has shown its effectiveness in test tasks and can be recommended for use as part of an electronic training polygon to test its capabilities in developing strategies for agents' behavior under conditions of uncertainty during their interaction.
A similar technology can be applied in automated process control support systems, where an operational assessment of the situation is necessary based on the available incomplete or inaccurate data with the development of recommendations for rational actions. In particular, when creating combat control systems, ensuring the safety of facilities, responding in emergency situations, as well as in control systems for unmanned vehicles.