A terminal control problem with linear dynamics and boundary-value optimization problem is considered. To solve the problem, a new gradient non-iterative approach, based on the necessary and sufficient condition of extremality, is proposed. The approach is reduced to solving a functional variational inequality on the set of controls in a Hilbert space. This inequality is treated as a parametric family of finite-dimensional variational inequalities that depend on time as a parameter. At each fixed moment of time, we have a cross-section of the problem in time, i.e. finite-dimensional variational inequality. Each such inequality has its solution, which is found using the continuous gradient projection method. We prove the pointwise convergence of the entire family of methods to their solutions. Thus, a mapping is formed, when the solution of the variational inequality corresponds to each instant of time. This function is the desired optimal control of the original problem.
x(t) = D(t)x(t) + B(t)u(t), x(t0) = x0, x (t1) = x1, x(·) ∈ ACn[t0, t1], u(·) ∈ U for almost all t ∈ [t0, t1], (1) (2) (3) Copyright ⃝c by the paper's authors. Copying permitted for private and academic purposes. where D(t), B(t) are n × n and n × r-continuous matrices (r < n). X1 is the reachability set, i.e. the set of right-hand ends x(t1) of trajectories. Controls u(·) ∈ U, where U ⊂ Rr is a convex closed set.
We understand any pair (x(·), u(·)) ∈ ACn[t0, t1] × U satisfying the condition ∫ t
t0 x(t) = x(t0) +
(D(τ )x(τ ) + B(τ )u(τ ))dτ, t0 ≤ t ≤ t1. as a solution to the differential system (1). Note that under these conditions the trajectory x(·) is an absolutely continuous function [Kolmogorov & Fomin, 2009]. The class of absolutely continuous functions is a linear variety that is everywhere dense in L2n[t0, t1]. We denote this class as ACn[t0, t1] ⊂ L2n[t0, t1], where the closure ACn[t0, t1] ≡ L2n[t0, t1]. For any pair (x(·), u(·)) ∈ ACn[t0, t1] × U both the Newton-Leibniz formula and, accordingly, the integration-by-parts formula are satisfied.1 In [Vasiliev, 2011, Book 2, p. 443] it is shown that in the linear differential system (1) for any control u(·) ∈ U there is a unique trajectory x(·), and this pair satisfies this identity (4).
In (1)-(3), it is necessary to find a control u (·) ∈ U such that the corresponding trajectory x (·), being a solution of the differential system, connects the initial x0 and the terminal x (t1) conditions.
The model (1)-(3) contains two components: controlled dynamics and the optimization problem as a boundaryvalue problem. The boundary-value problem is a model of a controlled object: for example, the enterprize – in the economy, the disease history – in the medicine, the assembly line – in technology, the epidemic or the war – in society, and so on. All these objects act in some environment, in which perturbations constantly arise. From time to time, under the influence of perturbations, objects lose their equilibrium state and find themselves in a random position. The problem arises: by choosing control, to return the object from an arbitrary state x0 ∈ Rn to the terminal state x (t1). 2
Classical Lagrangian and Original Problem The problem presented above is a problem of terminal control, formulated in a Hilbert space. In the theory of convex programming in finite-dimensional spaces, for a primal problem there always exists a dual problem in the dual space. Carrying out the corresponding analogies with finite-dimensional spaces, we consider elements of the duality theory in a functional Hilbert space using the example of a convex terminal control problem (1)-(3). To this end, we introduce the linear convolution of the problem, known as the Lagrange function: L(x(t1), x(·), u(·); ψ(·)) = φ(x(t1) + for all (x(t1), x(·), u(·)) ∈ X1 × ACn[t0, t1] × U, ψ(·) ∈ Ψ2n[t0, t1]. Here Ψ2n[t0, t1] is a linear variety of absolutely continuous functions in a space that is conjugate to the space L2n[t0, t1] of primal variable x(·). Since the space L2n[t0, t1] is a Hilbert space, it is self-adjoint one.
In order to take advantage of the duality theory, we linearize the Lagrange function (5) at the solution point of the problem (1)-(3), i.e. at the point (x (t1), x (·), u (·)). Since the Lagrange function has only one nonlinear term – the function φ(x(t1)), then we replace this term by its linear approximation from the expansion of φ(x(t1)) into a Taylor series. Then the linearized Lagrange function takes the form
L(x(t1), x(·), u(·); ψ(·)) = = ⟨∇φ(x (t1)), x(t1) − x (t1)⟩ + for all (x(t1), x(·), u(·)) ∈ X1 × ACn[t0, t1] × U, ψ(·) ∈ Ψ2n[t0, t1].
It is known [Ioffe & Tikhomirov, 1974] that the Lagrange functions for convex, regular (Slater’s condition holds) problems in finite-dimensional and functional spaces have saddle points (x1, x (·), u (·); ψ (·)). The first 1The scalar products and norms in the introduced spaces are de ned, respectively, as ⟨x(·); y(·)⟩ =n∫tt01 ⟨x(t); y(t)⟩dt; ∥x(·)∥2 = ∫tt01 |x(t)|2dt;
n ⟨x(t); y(t)⟩ = ∑ xi(t)yi(t); |x(t)|2 = ∑ xi2(t); t0 ≤ t ≤ t1;
1 1 x(t) = (x1(t); :::; xn(t))T; y(t) = (y1(t); :::; yn(t))T: (4) (5) (6) three components are the solution to the problem (1)-(3), and the last component plays the role of the Lagrange multiplier and, accordingly, is the solution to the dual (conjugate) problem. Further, using the linearization of the Lagrangian function (6), we obtain a dual problem in an explicit form.
By definition, the saddle point (x (t1), x (·), u (·); ψ (·)) satisfies the saddle-point system of inequalities common both for the Lagrange function (5) and for its linearization (6). We write out these saddle points of the inequality for the linearization (6) [Antipin, 2014], [Antipin & Khoroshilova, 2015(1)], [Antipin & Khoroshilova, 2015(2)], [Antipin & Khoroshilova, 2015(3)], [Antipin & Khoroshilova, 2016(1)], [Antipin & Khoroshilova, 2016(2)]:
⟨∇φ(x (t1), x (t1) − x (t1)⟩ + ≤ ⟨∇φ(x (t1)), x (t1) − x (t1)⟩ + ≤ ⟨∇φ(x (t1), x(t1) − x (t1)⟩ + for all (x(t1), x(·), u(·)) ∈ X1 × ACn[t0, t1] × U, ψ(·) ∈ Ψ2n[t0, t1]. We show now that the primal and dual components of the saddle point for the linearized Lagrange function (6) are primal and dual solutions to the original problem (1)-(3).
The left-hand inequality of (7) is the problem of maximizing a linear function in the variable ψ(·) on the entire linear variety Ψ2n[t0, t1]: ∫ t1 t0
d ⟨ψ(t), D(t)x (t) + B(t)u (t) − dt x (t)⟩dt ≤ d ⟨ψ (t), D(t)x (t) + B(t)u (t) − dt x (t)⟩dt. This inequality is true for all ψ(·) ∈ Ψ2n[t0, t1] only if
d
D(t)x (t) + B(t)u (t) − dt x (t) = 0, x (t0) = x0.
The right-hand inequality of (7) is the problem of minimizing the Lagrange function in the variables (x(t1), x(·), u(·)) if the function ψ(t) is fixed: ψ(t) = ψ (t). Let us show that the primal variables in (x (t1), x (·), u (·); ψ (·)) are the solution to (1)-(3).
In view of (9), from the right-hand inequality of (7), we have ⟨∇φ(x (t1)), x (t1)⟩ ≤ ⟨∇φ(x (t1)), x(t1)⟩ + for all (x(t1), x(·), u(·)) ∈ X1 × ACn[t0, t1] × U.
Considering the inequality (10) with an additional scalar constraint we get that the linear function ⟨∇φ(x1), x1⟩ reaches its minimum on the set defined by the scalar constraint (11). But, according to (9), the solution (x (t1), x (·), u (·)) belongs to a narrower set than (11). Therefore, this point remains a minimum on a subset of the solutions for the system (9), i.e.
t1 d ∫ ⟨ψ (t), D(t)x(t) + B(t)u(t) − dt x(t)⟩dt = 0, t0 ⟨∇φ(x (t1)), x (t1)⟩ ≤ ⟨∇φ(x (t1)), x(t1)⟩, d dt
x(t) = D(t)x(t) + B(t)u(t) for all (x(t1), x(·), u(·)) ∈ X1 × ACn[t0, t1] × U. Thus, if the Lagrange function (6) has a saddle point, then its vector of primal components is a solution to (1)-(3). (7) (8) (9) (10) (11) (12) (13) +
Dual Lagrangian and Dual Problem Let us show that the Lagrange function for linear dynamical problems allows to obtain the corresponding dual problems in conjugate spaces. Using the formulas for the transition to conjugate linear operators ⟨ψ(t), D(t)x(t)⟩ = ⟨DT(t)ψ(t), x(t)⟩, ⟨ψ(t), B(t)u(t)⟩ = ⟨BT(t)ψ(t), u(t)⟩ and the integration-by-parts formula on the interval [t0, t1] ⟨ψ(t1), x(t1)⟩ − ⟨ψ(t0), x(t0)⟩ = t0 ⟨ dt ψ(t), x(t)⟩dt + t0
d ⟨ψ(t), dt x(t)⟩dt, we write out the Lagrangian conjugate with respect to (6):
LT(ψ(·); x(t1), x(·), u(·)) = ⟨DT(t)ψ(t) + ψ(t), x(t)⟩dt + ⟨BT(t)ψ(t), u(t)⟩dt + ⟨ψ0, x0⟩ (14) for all ψ(·) ∈ Ψ2n[t0, t1], (x(t1), x(·), u(·)) ∈ Rn × ACn[t0, t1] × U, where ψ1 = ψ(t1).
The saddle point (x (t1), x (·), u (·); ψ1 , ψ (·)) satisfies the saddle-point system (7), which in the conjugate space has the form
⟨∇φ(x (t1)) − ψ1, x (t1)⟩ + ⟨ψ0, x0⟩ ⟨DT(t)ψ(t) + ψ(t), x (t)⟩dt +
⟨BT(t)ψ(t), u (t)⟩dt + ⟨ψ0, x0⟩ ≤ ⟨∇φ(x (t1)) − ψ (t1), x (t1)⟩ + ⟨ψ0 , x0⟩ ≤ ∇φ(x (t1)) − ψ (t1), x(t1)⟩ + ⟨ψ0 , x0⟩ ⟨DT(t)ψ (t) + ψ (t), x (t)⟩dt +
⟨BT(t)ψ (t), u (t)⟩dt for all (x(t1), x(·), u(·)) ∈ X1 × ACn[t0, t1] × U, ψ1 ∈ Rn, ψ(·) ∈ Ψ2n[t0, t1].
We will repeat the same transformations as in the previous section, where it was shown that the initial problem follows from the saddle-point system. But now we will get the dual problem.
From the right-hand inequality of (15), we have ⟨∇φ(x (t1)) − ψ (t1), x (t1) − x(t1)⟩ + ⟨DT(t)ψ (t) +
ψ (t), x (t) − x(t)⟩dt+ d dt ⟨BT(t)ψ (t), u (t) − u(t)⟩dt ≤ 0 t0 for all (x(t1), x(·), u(·)) ∈ X1 × ACn[t0, t1] × U. By virtue of the independent change of each of the variables (x(t1), x(·), u(·)) within its admissible subspaces (sets), the last inequality is decomposed into three independent inequalities: ⟨∇φ(x (t1)) − ψ (t1), x (t1) − x(t1)⟩ ≤ 0,
x(t1) ∈ Rn, ⟨DT(t)ψ (t) + ψ (t), x (t) − x(t)⟩dt ≤ 0,
x(·) ∈ ACn[t0, t1], ⟨BT(t)ψ (t), u (t) − u(t)⟩dt ≤ 0,
u(·) ∈ U.
t0 Both first linear functionals reach a finite extremum on the entire subspace only in the case when their normals vanish, which leads to a system of problems
DT(t)ψ (t) + d dt ψ (t) = 0, ∇φ(x (t1)) − ψ (t1) = 0, (16) + The left-hand inequality in (15), taking into account (16) and (17), can be rewritten as ⟨BT(t)ψ (t), u (t) − u(t)⟩dt ≤ 0, ∀u(·) ∈ U. We require additionally that the first two terms of this inequality satisfy conditions ⟨∇φ(x (t1)) − ψ(t1), x (t1)⟩ ≥ 0, d dt ψ(t), x (t)⟩dt ≥ 0.
Then the initial inequality can be represented as an optimization problem of the form (ψ (·), ψ(t1)) ∈ Argmax
⟨BT(t)ψ(t), u (t)⟩dt | ⟨∇φ(x (t1)) − ψ(t1), x (t1)⟩ ≥ 0, Hence, taking into account (16), we have where ψ(·) ∈ Ψ2n[t0, t1].
Combining with (17),(18), we get the problem dual with respect to (1)-(3):
⟨BT(t)ψ(t), u (t)⟩dt ≤ ⟨BT(t)ψ (t), u (t)⟩dt + ⟨ψ0 , x0⟩.
t0 ≤ d dt d dt d dt We consider together the left-hand inequality of the saddle-point system (7) for the classical Lagrangian and the right-hand inequality of the dual saddle-point system (15) for the conjugate Lagrangian. Of these systems, partial subsystems (9) and (16),(17) were obtained as a consequence. We write them out and arrive at the following boundary-value problem: (ψ (·), ψ(t1)) ∈ Argmax
DT(t)ψ(t) + ∫ t1 t0 {∫ t1 t0 ⟨BT(t)ψ(t), u (t)⟩dt |
} ψ(t) = 0, ψ(t1) = ∇φ(x (t1)) , d dt d dt (18) (19) (20) (21) (22) (23) ∫ t1
t0 ∫ t1 t0 d dt d dt d dt d dt We represent the equation (30) in the form of a variational inequality
d d ⟨ dt u(t) + αBT (t)ψ(t), z(t) − u(t) − dt u(t)⟩ ≥ 0 u (t) = πU(u (t) − αBT(t)ψ (t)), t0 ≤ t ≤ t1,
x(t) = D(t)x(t) + B(t)u(t), x(t0) = x0, ψ(t) + DT(t)ψ(t) = 0, ψ(t1) = ∇φ(x(t1)), where πU(·) is the projection operator on the set of controls U, α > 0. 5
Continuous Method for Solving the Boundary-Value Differential System In order to solve the system (22)-(24), which is a sufficient extremality condition for the problem (1)-(3), we can formulate the gradient projection method. In our situation, it looks like this d u(t) + u(t) = πU(u(t) − αBT(t)ψ(t)). (30) dt
The process (28)-(30) is a family of finite-dimensional continuous gradient projection methods, each of which converges to the solution of the problem for a given t ∈ [t0, t1] [Antipin, 1994], [Liao, 2005]. In general, the whole family converges point-by-point.
Theorem. If the set of solutions (x (t1), x (·), u (·); ψ (·)) to the problem (28)-(30) is not empty and belongs to the space X1 × ACn[t0, t1] × U × Ψ2n[t0, t1], the terminal function is convex, U is strictly convex closed set, then the family of continuous gradient projection methods converges pointwise to the control u (·) ∈ U as t → ∞.
Proof. We represent (29) and (26) in the form of variational inequalities ⟨∇φ(x(t1)) − ψ1, x (t1) − x(t1)⟩ + ⟨DT (t)ψ(t) + ψ(t), x (t) − x(t)⟩dt ≥ 0, ⟨∇φ(x (t1)) − ψ1 , x (t1) − x(t1)⟩ + ⟨DT (t)ψ (t) + ψ (t), x (t) − x(t)⟩dt ≥ 0, (31)
t0
The variational inequality of this system can be rewritten in the equivalent form of the operator equation with the projection operator on the corresponding convex closed set U [Vasiliev, 2011]. Then we obtain a system of differential and operator equations: (24) (25) (26) (27) (28) (29) (32) (33) add together both inequalities, then Using the integration by parts formula from Section 3, we transform (28) and obtain ⟨∇φ(x(t1)) − ∇φ(x (t1)), x (t1) − x(t1)⟩ − ⟨ψ1 − ψ1 , x (t1) − x(t1)⟩ + ⟨DT (t)(ψ(t) − ψ (t)) + (ψ(t) − ψ (t)), x (t) − x(t)⟩dt ≥ 0. Adding the last two inequalities and combining the result with the inequality (32), we obtain ⟨BT(t)ψ (t), u (t) − u(t)⟩dt − ⟨BT(t)ψ (t),
u(t)⟩dt ≤ 0. 1 d 2 dt |u(t) − u (t)|2 + d dt u(t)
2 +α⟨BT (t)(ψ(t) − ψ (t)), u(t) − u ⟩ + α⟨BT (t)(ψ(t) − ψ (t)), d dt u(t)⟩ ≤ 0, ∫ t1 t0
d ⟨ψ(t) − ψ (t), D(t)(x (t) − x(t)) − dt (x (t) − x(t))⟩dt ≥ 0. 1 d 2 dt |u(t) − u (t)|2 + d dt u(t) + α⟨BT (t)(ψ(t) − ψ (t)), d dt u(t)⟩ ≤ 0, for all z(·) ∈ U. We put z(t) = u (t) in (33), then we get d dt d dt Taking into account that u(t) =
(u(t) − u (t)), from (33) we have d d ⟨ dt u(t) + αBT (t)ψ(t), u (t) − u(t) − dt u(t)⟩ ≥ 0.
1 d −⟨ 2 dt (u(t) − u (t)), u(t) − u (t)⟩ − d dt u(t) + α⟨BT (t)ψ(t), u − u(t)⟩ − α⟨BT (t)ψ(t), d dt u(t)⟩ ≥ 0. 1 d 2 dt |u(t) − u (t)|2 + d dt
2 u(t) + α⟨BT (t)ψ(t), u(t) − u (t)⟩ + α⟨BT (t)ψ(t), d dt u(t)⟩ ≤ 0.
Hence, we find α
The last term on the right-hand side of the inequality is zero by virtue of (22) and (28). The second term can be transformed to the form of the total derivative in the direction, i.e. ddt F (x(t)) = ddx F (x) · ddt x(t). In view of the foregoing, the inequality (39) can be represented in the form 1 d 2 dt |u(t) − u (t)|2 + d dt
2 u(t) + α d dt
F (x) ≤ 0.
From here
2 1 u(t) dt + αF (x(t1)) ≤ 2 |u(t0) − u (t0)|2 + αF (x(t0)). (34) (35) (36) (37) (38) (39) which we also denote by ti such that u(ti) → u′ and in this subsequence as ti → ∞, we obtain d x′ (t) = D(t)x′ (t) + B(t)u′ (t), x′ (t0) = x0, dt d ψ′ (t) + DT(t)ψ′ (t) = 0, ψ′ (t1) = ∇φ(x′ (t1)), dt u′ (t) = πU (u′ (t) − αBT(t)ψ′ (t)). (40) (41) (42) Comparing this system with (22), one can see that the solution obtained is the primal and dual solutions to the original problem: (x′ (t1), x′ (·), u′ (·); ψ′ (·)) = (x (t1), x (·), u (·); ψ (·)). The theorem is proved. Acknowledgements This work was supported by the Russian Foundation for Basic Research (project 15-01-06045) and by the Program of State Support of Leading Scientific Schools (project NSh-8860-2016.1).