Introduction

In recent years, an agent-oriented approach has been successfully developed within the framework of the theory of artificial intelligence. Agents are considered rational or intelligent and can form multi-agent systems and act to achieve a common goal. Agent systems are studied from different points of view: from the standpoint of the theory of conflict processes, information theory, social psychology, software engineering, in the context of concepts of electronics. The proposed paper considers the aspect of optimizing the actions of agents as part of a multi-agent system.

The paper [1] shows that the main types of activities related to the management of systems of rational agents and individual agents are cooperation (formation of agent systems), planning and coordination of agent actions, system placement and recognition. Problems of cooperation, planning and coordination of agents' actions are studied in detail in the monograph [2]. Recognition tasks are studied in [3 -6], placement tasks are studied in papers [7,8]. In [9], the concept of a rational agent is defined and used.

To solve optimization planning problems, that is, to find optimal action plans in multi-agent systems consisting of rational agents, mathematical programming methods are used, in particular methods of linear, nonlinear, stochastic, discrete programming [10 -14]. Multi-criteria optimization problems and corresponding mathematical methods are important for multi-agent systems [15,16]. In paper [16], human-machine procedures are used to solve multi-criteria problems, in which the method of nonlinear programming is combined with the work of a DM. The difficulty of this approach is that the DM is not able to make decisions multiple times under the condition of a significant number of iterations of the nonlinear programming method.

This paper considers the problem of optimal planning of actions of a multi-agent system consisting of rational agents. Planning optimization problems and corresponding mathematical methods for their solution are described. In multi-criteria optimization problems, it is proposed to use an artificial neural network instead of DM, and methods of determining the optimal values of the parameters of such a network are proposed.

Agents and multi-agent systems

The concept of a rational agent is widely used in the theory of artificial intelligence, economics, game theory and has a unifying meaning, allowing to define the main tasks facing agents and multiagent systems, the relationships between these tasks, etc. In [9], an autonomous object that perceives the environment with the help of sensors and influences this environment with the help of executive mechanisms is considered an agent. The agent's program works in a computing device and receives data from sensors, recognizes and analyzes the data, calculates the optimal strategy of the agent's behavior, and issues commands to executive mechanisms. An agent can be a computer program, a robot, or a person. The following definition of a rational agent is given in [9]: "For each possible perceptual sequence, a rational agent must choose the actions that are expected to maximize its performance, given the facts provided by the perceptual sequence and all the built-in knowledge the agent has".

Usually, there are different permissible sequences of actions of an agent that lead to a goal. In this case, it is appropriate to consider that there is a utility function defined on a set of action sequences of an agent (or a system of agents), which takes values from a set of real numbers. A rational agent is an agent who, for the sake of achieving a goal, uses the optimal behavior strategy, maximizing the utility function. A multi-agent system is a system consisting of rational agents who have a common goal and use an optimal strategy to achieve it. It can be assumed that an optimization problem is formulated for the system and agents form a behavior strategy using the optimal solution of this problem. Let us give examples of systems consisting of rational agents:

• the system of state administration bodies, enterprises, institutions and organizations of the country's defense-industrial complex; • the system of economies of different countries developing in cooperation;

• a group of drones chasing a target. Note that the system of agents may include a central agent that performs some of the management functions.

In the literature, there are definitions of agents of other types that differ from rational agents. In [2], the concept of an intelligent agent is defined, which should have the following properties:

• reactivity, i.e. the ability to perceive the state of the surrounding environment and act accordingly; • proactivity, i.e. the ability to identify one's own initiative;

• social activity, i.e. the ability to interact with other agents to achieve a goal.

Agents are considered autonomous. Autonomy means that the agent's behavior is determined not only by the environment, but also to a large extent by the properties of the agent.

The definition of an agent is called "weak" if it contains only the described features, that is, autonomy, reactivity, proactivity, social activity [17]. If, in addition to the above signs, there are additional ones in the definition of the agent, then the definition is called "strong". Additional properties may include:

• desires, i.e. situations desirable for the agent;

• intentions, i.e. what needs to be done to satisfy desires or to fulfill obligations to other agents; • goals, i.e. a set of intermediate and final goals of the agent;

• obligations to other agents; • knowledge, i.e. a part of knowledge that does not change during the agent's existence;

• beliefs, i.e. a part of the agent's knowledge that can change.

We assume that agents have the above and possibly other properties to the extent necessary to solve problems and use the resulting solutions.

Planning is the development of a method of action of a multi-agent system and individual agents in the future depending on the situations that may arise, the choice of an effective method of action, optimal allocation of resources. Centralized planning, distributed development of a centralized plan, distributed development of a distributed plan are possible [18]. Centralized planning is performed by a dedicated agent that has the necessary resources and information.

Distributed development of a distributed plan means that there is no dedicated agent, agents build individual plans interacting with each other. In this case, it is considered that the agents have limited computing capabilities, information about the local environment, and limited communication capabilities. Examples include groups of autonomous vehicles, mobile sensor networks, routing in data networks, transportation systems, multiprocessor computing, energy systems [19].

Central planning is discussed next.

Optimization planning models in multi-agent systems and corresponding numerical methods

Consider optimization problems of planning in multi-agent systems. Such problems can be applied, in particular, in the process of planning the activities of the defense-industrial complex of Ukraine. The defense-industrial complex is a system consisting of interrelated research centers, state organizations, and industrial enterprises that produce goods for military purposes. We consider each element of this system to be a rational agent. Suppose there is a dedicated agent (for example, a ministry) that has the information needed for planning.

Let the system contain m rational agents that produce several types of products. Some agents may not produce final products while performing managerial functions. Let n be the number of types of products, ij x denote the value of the j -th product produced by the i -th agent, let j α be the given specific weight of the j -th product in the set of final products, ). ,..., , (

..., 1 x j m i ij n j x → ∑ = = α (1) subject to , ,..., 1 , ,..., 1 , 1 l k m i b x a n j ik ij ijk = = ≤ ∑ = (2) . ,..., 1 , ,.., 1 , max min n j m i x x x ij ij ij = = ≤ ≤ (3)

Here ijk a is the cost of the k -th resource for the production of the j -th product of unit value at the i -th enterprise, ik b is the stock of the k -th resource at the i -th enterprise, l is the number of types of resources, max min , ij ij x x are the minimum and maximum permissible quantities of the j -th product produced at the i -th enterprise, . 0 max min

∞ < ≤ ≤ ij ij x x

Let X be the set of vectors x that satisfy conditions (2), (3). Using an additional variable , y we write the problem ( 1) -( 3) as follows: 4) -( 6) is a linear programming problem and is solved by linear programming methods. Note that the numbers appearing in problems ( 1) -( 3) and ( 4) -( 6) represent part of the agents' knowledge and beliefs, and the objective function expresses the intentions of the system.

max, , y x y → (4) , ,..., 1 , 1 n j y x j m i ij = ≥ ∑ = α (5) . X x ∈ (6) Problem (

Suppose, in the problem ( 1) -( 3), instead of the objective function (1), some objective function

) (x f is used. It is necessary to solve the problem . max ) ( X x x f ∈ → (7)

We consider the function ) (x f to be concave (that is, convex upwards) on the set X and smooth on some neighborhood of the set X . Smoothness means that the function is defined and has continuous partial derivatives with respect to all variables, and an open set containing the X is called a neighborhood of X . To solve this problem, it is advisable to use the conditional gradient method (Frank and Wolf method) [10,11], which consists of the following. Let's choose

. 0 X x ∈ At the s -th step ( ,... 2 , 1 , 0 = s ) we calculate , ), ( max arg x x f x s X x s ∇ = ∈ (8) )), ( ( max arg 1 0 s s s t s x x t x f t − + = ≤ ≤(9)

). ( x the optimal solution of problem (7). The following inequality holds . ), ( ) ( ) (

s s s s x x x f x f x f − ∇ ≤ − * (11)

Indeed, the concavity of the function

) (x f implies an inequality . ), ( ) ( ) ( s s s x x x f x f x f − ∇ ≤ − * * Obviously, . ), ( ), ( s s s x x f x x f ∇ ≤ ∇ *

Last two inequalities imply (11).

The partial boundaries of the sequence } { s

x are the maximum points of the function ) (x f on the set .

X In [10] it is proved that

); / 1 ( ) ( ) ( s O x f x f s = − * (12)

this estimate cannot be improved. It follows from (12) and the results of [12] that the conditional gradient method is not suboptimal. It is advisable to use this method for solving problem (7) due to the possibility of applying estimate (11), which allows stopping the computational process after reaching the required accuracy, and due to its simplicity and satisfactory speed of convergence in practice. The conditional gradient method is used in the next section as a component of multicriteria optimization.

If the function f is not smooth or instead of the gradient (generalized gradient) of the function f its stochastic analogue is known, non-smooth optimization and stochastic programming methods [11 -13] are used to solve problem (7). If, instead of problem (7), a discrete programming problem is considered, branch-and-bound methods, heuristic algorithms, random search methods with local optimization, and polynomial approximate schemes can be used [14].

The system of economies of different countries interacting with each other through the export and import of goods, services, and capital can be considered a system of rational agents, provided that the governing bodies of the countries make rational economic decisions in market conditions. There are problems of optimizing the interaction of a separate economy with other economies and with the world market. Corresponding mathematical models and numerical methods are considered in works [20,21].

Multi-criteria optimization and human-machine procedures

In addition to a common goal, each rational agent of a multi-agent system can have its own optimality criterion. The problem is to find a quality solution that satisfies all agents. Let the extremal problem for the i -th agent have the form max, ) (

X x i x f ∈ → (13) where }, ,..., 1 { , m I I i = ∈ ), ,..., ( 1 n x x x =

X is a convex closed bounded set of n -dimensional Euclidean space. We assume that the criteria are non-negative and normalized, that is, each criterion

) (x f i is replaced by , / ) ( * i i f x f where ). ( max * x f f i X x i ∈ =

To solve a multi-criteria extremal problem, the convolution method can be applied, replacing the set of criteria ( 13) with a single criterion. Such replacement can be performed in several ways [15]:

• The minimum of values

) (x f i i α is maximized: . max ) ( min X x i i I i x f ∈ ∈ → α • The weighted sum of values ) (x f i is maximized: . max ) ( 1 X x m i i i x f ∈ = → ∑ α • Product is maximized: . max ) ( 1 X x m i i i x f ∈ = → ∏ α

Here i α are given non-negative numbers, . 1

1 ∑ = = m i i α

The method of the main criterion, the method of objective programming and the method of concessions are described in [15]. The main criterion method consists of maximizing one selected criterion, all others are considered bounded from below: max, ) (

x k x f → }, { \ , ) ( min k I i f x f i i ∈ ≥ . X x ∈

Here min i f are given numbers, .

I k ∈

The objective programming method consists of setting the desired values of criteria m f f ,..., 1 and solving an extremal problem

. min ) ( / 1 1 X x p m i p i i i f x f ∈ = → ⎟ ⎠ ⎞ ⎜ ⎝ ⎛ − ∑ α

Here 1 ≥ p is given number. In the concession method, the criteria are numbered in order of decreasing importance. In the first step, the first criterion is maximized,

), ( max 1 1 * 1 x f f X x∈ = where . 1 X X = In the second step, the set } ) ( , : { 1 * 1 1 1 2 Δ − ≥ ∈ = f x f X x x X

is determined, where 1 Δ is the amount of concession for the first criterion, and the problem

) ( max 2 2 * 2 x f f X x∈ = is solved. In the third step, the set } ) ( , : { 2 * 2 2 2 3 Δ − ≥ ∈ = f x f X x x X

is determined, where 2 Δ is the concession amount for the second criterion, and the problem

) ( max 3 3 * 3 x f f X x∈ = is solved. The process continues until the last m -th step.

Consider the human-machine method of multi-criteria optimization [16]. This method uses a non-linear programming method, for which the directions of movement and step sizes are determined using a DM. We will consider the problem of multi-criteria optimization as follows:

. max )) ( ),..., ( ( ) ( 1 X x m x f x f u x f ∈ → = (14)

Here u is the utility function, X is a convex closed bounded set of n -dimensional Euclidean space. We assume that the function ) (x f i s concave on X . The following conditions [16] are sufficient for the concavity of the function f on the set X :

• the function u is concave, and the functions , ,..., 1 , m i f i = are linear;

• the functions u and , ,..., 1 , m i f i = are concave, and the function u is nondecreasing in each variable.

Let us assume that the conditions are fulfilled that guarantee the existence of continuous partial derivatives of the function

)) ( ),..., ( ( 1 x f x f u m

and the validity of the rule for their calculation [22]. To solve problem ( 14), we will use the conditional gradient method ( 8) - (10). According to the rule for calculating partial derivatives, we have

. ) ( )) ( ),..., ( ( ) ( 1 1 ∑ = ∇ ⎟ ⎟ ⎠ ⎞ ⎜ ⎜ ⎝ ⎛ ∂ ∂ = ∇ = ∇ m i s i s i s m s s x f f u x f x f u x f Here, ( ) − ∂ ∂ s i f u the i -th partialx x f w x m i s i s i X x s ∑ = ∈ ∇ =(15) where ( ) ( ) . 1 ss i s i f u f u w ∂ ∂ ∂ ∂ =

In (15), all quantities, except for , s i w are considered known. The values of , s i w showing the relative importance of the first and i -th criteria, are determined using DM, after which optimization ( 15) is performed. In [16], two ways of determining s i w are proposed. Suppose the function u satisfies the conditions of the theorem on the total increment of a function of several variables [22]. Let us assume that at point

)) ( ),..., ( ( 1 s m s x f x f criterion 1 f received an increment , 1 Δ criterion − i f an increment , i

Δ and the values of the remaining criteria did not change. We have ), ( ),..., ( , ) ( (

1 2 1 1 s i s s x f x x f u u − Δ + = Δ ( ) + Δ ∂ ∂ = − Δ + + 1 1 1 1 )) ( ),..., ( ( )) ( ),..., ( , ) ( s s m s s m s i i s i f u x f x f u x f x f x f ( ) ( ). ) ( ) ( 2 2 1 i i s i o f u Δ + Δ + Δ ∂ ∂ + Let DM choose the values i Δ Δ , 1 so that . 0 = Δu

In this case, we have approximately

Δ ∂ ∂ + Δ ∂ ∂ i s i s f u f u that is . 1 i s i w Δ Δ − =( ) ( ) , 0 1 1 =

The second way of calculating s i w is based on the fact that the gradient of a function indicates the direction of its fastest growth. Let all criteria except the first and i -th remain unchanged, the first and i -th grow by amounts

1 0 1 ≤ ≤ → − + − + t s s s m s s s x x t x f x x t x f u (18)

Here, the objective function is a function of one variable t . One graph shows the curves )), ( ( In [16], an example of the application of the described human-machine method of multi-criteria optimization to the organization of the educational process at one of the university faculties is given. Human-machine procedures can be combined not only with the conditional gradient method, but also with other methods of nonlinear programming.

s s s j x x t x f − + , ,..., 2 , 1 m j =

Obviously, DM cannot always qualitatively perform the tasks assigned to it in the process of multi-criteria optimization. In addition, the nonlinear programming method can perform a large number of iterations, which is unacceptable for DM. Therefore, it is advisable to use an artificial intelligence system instead of DM. In particular, it is possible to use an artificial neural network that calculates the values of , s i w using formulas ( 16) and (17), or calculates the gradient of the function f in another way, and the optimal value of t in problem (18). Such networks are considered in the next section

Use of artificial neural networks in optimization methods

Artificial neural networks (hereinafter referred to as neural networks) have become widespread as part of artificial intelligence systems. We consider a function weighting coefficients. The scheme of an artificial neuron is shown in Figure 1.

A neural network combining a large number of artificial neurons is capable of solving complex problems. We assume that the network has n inputs and one output and calculates some function We call a function σ The number of artificial neurons in such a neural network is not limited. If you choose a sigmoid function

∑ ∑ + = = ⎟ ⎟ ⎠ ⎞ ⎜ ⎜ ⎝ ⎛ = n i n j j ij i n x x x f ψ ϕ (19)) (t σ sigmoid if the condition ⎩ ⎨ ⎧ −∞ → +∞ → → t t t , 0 , ,1

), (t σ that has a continuous derivative, then the quantity G will have continuous partial derivatives for all quantities