Theoretical Bound of the Complexity of Some Extragradient- Type Algorithms for Variational Inequalities in Banach Spaces

Abstract

The work presents a study of three new extragradient-type algorithms for solving variational inequalities in a Banach space. Two algorithms are natural modifications of Tseng method and “Extrapolation from the Past” method for problems in Banach spaces, based on the generalized Alber projection. The third algorithm, called the operator extrapolation method, is a variant of forward-reflected-backward algorithm, where the generalized Alber projection is also used instead of the metric projection onto the admissible set. Advantage of the latter algorithm is only one calculation of the operator value and the generalized projection onto the feasible set on each iteration. For variational inequalities with monotone Lipschitz operators acting in a 2uniformly convex and uniformly smooth Banach space O  1  estimates for the complexity in terms of the gap function are proved.

Keywords

• 1 variational inequality
• monotone operator
• extragradient algorithm
• Extrapolation from the Past
• gap function
• complexity
• 2-uniformly convex Banach space
• uniformly smooth Banach space

1. Introduction

n where p 1  i1 pi . The last saddle point problem can be rewritten as a bilinear minmax problem on the product of standard simplexes min max  Jv, Px  x , xn v2n where J is an n  2n -matrix of the form  I

I  , where I is the identity matrix. We got the following variational inequality problem

find x  n , v  2n :  P*  I  J v , x  x    J *  I  P x,v  v   0 x  n v  2n .

Note that often nonsmooth optimization problems can be effectively solved with algorithms for variational inequalities, if former are reformulated as saddle point problems [ 6 ]. With the growing popularity of generative adversarial networks (GANs) and other adversarial learning models, a steady interest in algorithms for solving variational inequalities has arisen among specialists in the field of machine learning [ 7, 8 ].

The most widely known method for solving variational inequalities is so called Korpelevich extragradient algorithm [ 9 ]. Many publications are devoted to the study of this algorithm and its modifications [ 6, 10–16 ].

An effective modern version of the extragradient method is Nemirovski mirror-prox method [ 6 ]. This method can be interpreted as a variant of the extragradient method with projection understood in the sense of Bregman divergence. One more interesting method of dual extrapolation for solving variational inequalities was proposed by Nesterov [ 10 ]. Adaptive variants of the Nemirovski proximal mirror method were studied in [ 11–13 ].

In the early 1980s, Popov proposed a modification of the classical Arrow-Hurwitz algorithm for finding saddle points of convex-concave functions [ 17 ]. Recently Popov's algorithm for variational inequalities has become well known among machine learning specialists under the name “Extrapolation from the Past” [ 7, 8 ].

A modification of Popov's method for solving variational inequalities with monotone operators was studied in [ 18 ]. And in the article [ 19 ], a two-stage proximal algorithm for solving the equilibrium programming problem is proposed, which is an adaptation of the method [ 17 ] to the general Ky Fan inequalities. The algorithm from [ 19 ] uses Bregman divergence instead of Euclidean distance. Further development of this circle of ideas led to the emergence of the so-called forward-reflected-backward algorithm [ 20 ]: and similar method [ 21 ]: xn1  PC  xn n Axn n1  Axn  Axn1  , xn1  PC  xn  n Axn   n1  Axn  Axn1  .

Recently, using theory of Banach spaces and constructions of their geometry [ 26–30 ], progress has been achieved in the research of algorithms for problems above in Banach spaces [ 3, 23–25 ]. Extensive material on this topic is contained in the book [ 3 ]. The next algorithm for solving variational inequalities in a 2-uniformly convex and uniformly smooth Banach space was proposed in [ 22 ]: xn1  C J 1  Jxn   Axn  ,  yn  C J 1  xn  n Axn ,  xn1  J 1  Jyn  n  Ayn  Axn , where С is Alber generalized projection operator [ 23 ],   0 , J is normalized dual mapping from E to E* . This method weakly converges for inversely strongly monotone (cocoercive) operators A : E  E* . Shehu [ 24 ] has recently extended Tseng’s result to 2-uniformly convex and uniformly smooth Banach spaces. He proposed the next weakly converging process: where n  0 is either chosen based on operator A Lipschitz constant value or calculated with a kind of linear search procedure.

It should be noted that the early research on algorithms for solving variational inequalities was usually concentrated on the study of convergence of algorithms and related questions of an asymptotic nature [ 13–15, 17, 19–25 ].

More recent studies are focused on estimating the number of iterations of algorithms required to obtain an approximate solution of a given quality [ 6, 8, 10–12, 16, 18 ]. This direction of research was initiated by the work of Nemirovski [ 6 ], where mentioned earlier mirror-prox algorithm was proposed and O  1  complexity estimate in terms of the gap function was obtained for the class of problems with monotone Lipschitz continuous operator.

A fundamental question arose about the construction of an algorithm with O  1  complexity estimation and single computation of the operator's value and projection onto the feasible set at the iteration step. Algorithm [ 20 ] answers this question in the case of a Hilbert space.

This work is devoted to the study of three new extragradient type algorithms for solving monotone variational inequalities in a Banach space. The first two algorithms are natural modifications of Tseng's method [ 14 ] and “Extrapolation from the Past” method [ 18 ] for problems in Banach spaces using the generalized Alber projection. Iteration of each of these algorithms is more economical than iteration of the extragradient method, because the first one uses single projection on iteration, and the second one needs only one operator calculation. The third algorithm, called operator extrapolation method, is a variant of the forward-reflected-backward algorithm, proposed by Malitsky and Tam [ 20 ]. Operator extrapolation method also uses generalized Alber projection onto the feasible set. An attractive feature of the algorithm is only one computation at the iterative step of the operator value and the generalized projection. The O  1  complexity estimations are proved in terms of the gap function for variational inequalities with monotone Lipschitz operators acting in a 2-uniformly convex and uniformly smooth Banach space.

The article has the following structure. Section 2 contains the necessary information on the geometry of Banach spaces. Section 3 is devoted to variational inequalities. The algorithms are described in Section 4. The formulations and proofs of complexity estimations are presented in Section 5.

2. Preliminaries

Let us remind some basic terms and results from Banach spaces geometry, which are needed for us to formulate and prove our results [ 23, 25–30 ].

Everywhere later E is a real Banach space with norm  , E – dual space for E , x , x – value of functional x  E on element x  E . Let’s also denote   norm in E .

Let SE  x  E : x 1 . Banach space E is called strictly convex, if for all x, y  SE and x  y Banach space E is called uniformly convex, if  E    0  0, 2 [ 26, 27 ]. Banach space E is called 2-uniformly convex, if exists c  0 such that  E    c 2 for all  0, 2 [ 27 ]. Obviously 2uniformly convex space is uniformly convex. It’s known that uniformly convex Banach space is reflexive [ 26-28 ].

Banach space E is called smooth, if lim t0 x  ty  x

t exists for all x, y  SE [ 26 ]. Banach space E is called uniformly smooth if limit (1) exists uniformly for x, y  SE [ 26 ]. There is a duality between convexity and smoothness of E and it’s dual space E (1) [ 26,27 ]: E – strictly convex  E – smooth; E – smooth  E – strictly convex; E – uniformly convex  E – uniformly smooth; E – uniformly smooth  E – uniformly convex. The first two implications can be reversed for reflexive space E .

Smoothness module of space E is defined as Uniform smoothness of Banach space E is equivalent to the relation limt0  E t t1  0 [ 27, 28 ]. Banach space E is called 2-uniformly smooth, if exists c  0 such that  E t   ct2 for all t  0 [ 27, 28 ]. Banach space E is 2-uniformly convex if and only if E is 2-uniformly smooth [ 27–29 ].

It is known, that Hilbert spaces and spaces Lp (1  p  2 ) are 2-uniformly convex and uniformly smooth ( Lp are uniformly smooth for p 1,  and 2-uniformly smooth for p 2,  ) [ 27, 28 ].

Multivalued mapping J : E  2E , which has the form

Jx  x  E : x, x  x 2  x 2 ,  is called normalized dual mapping [ 27, 28 ].

For Hilbert space J  I (identity). It is known [ 23, 27, 28 ] that: if space E is smooth, then mapping J is single-valued; if E is strictly convex, then J is injective and strictly monotone; if space E is reflexive, then mapping J is surjective; if E is uniformly smooth, then J is uniformly continuous on bounded subsets of E . The explicit form of the mapping J for spaces p , Lp and Wpm ( p 1, ) is given in [ 3, 23, 27, 28 ].

Let E be a smooth Banach space. Let’s consider functional, introduced by Y. Alber in [ 23 ]: d  x, y  x 2  2 Jy, x  y 2 x, y  E.

The next useful 3-point identity follows from the definition above: d  x, y  d  x, z   d  z, y  2 Jz  Jy, x  z x, y, z  E.

If space E is strictly convex, then for x, y  E we have

For Banach spaces

p , Lp and Wpm (1  p  2 ) we have   [ 29, 30 ]. For a Hilbert space, inequality from Lemma 1 becomes identity with   1.

Lemma 2 ([ 29 ]). Let E be a 2-uniformly smooth Banach space. Then for some   0 x  y 2  x 2  2 Jx, y  y 2 x, y  E .

For Banach spaces p , Lp and Wpm ( 2  p   ) we have   p 1 [ 29, 30 ]. For a Hilbert space, inequality from Lemma 2 becomes identity with  1 .

Let K be a non-empty closed and convex subset of a reflexive, strictly convex and smooth space E . It is known [ 23 ] that for each x  E there exists unique point z  K such that 1 p 1 d  z, x  inf d  y, x .

yK d  x, y  0  x  y .

1  d  x, y  

x  y 2 x, y  E .

Lemma 1 ([ 25 ]). Let E be a 2-uniformly convex smooth Banach space. Then for some   1

This point z is denoted by  K x , and the corresponding operator K : E  K is called generalized projection of E onto K (generalized Alber projection) [ 23 ]. Note that for a Hilbert space  K coincides with the metric projection onto the set K .

Lemma 3 ([ 23 ]). Let K – closed and convex subset of a reflexive, strictly convex and smooth space E , x  E , z  K . Then

Inequality from Lemma 3 is equivalent to the next one [ 23 ]: z  K x 

Jz  Jx, y  z  0

y  K .

d  y,K x  d K x, x  d  y, x y  K .

Remark 1. The main element of the algorithms studied below is the calculation of a new point x  K J 1  Jx  x  by known x  E and x  E . From Lemma 3 and mentioned 3-point identity follows the inequality fundamental for the analysis of algorithms.

d  y, x   d  y, x  d  x , x 2 x, y  x y  K .

Basic information about monotone operators and variational inequalities in Banach spaces can be found in [ 1, 3, 23, 28 ].

3. Variational inequalities

Let E be a 2-uniformly convex and uniformly smooth Banach space, C is a non-empty subset of space E , A is an operator, acting from E to E* . Let’s consider the variational inequality: find x  C :

Ax, y  x  0 y  C .

Let’s denote set of solutions of (2) as S .

We need the next assumptions:  set C  E is convex and closed;  operator A : E  E* is a monotone and Lipschitz continuous with constant L  0 on C ;  set S is non-empty.

Let’s consider dual variational inequality: find x  C :

Ay, x  y  0 y  C . was studied in [ 22 ] for inversely strongly monotone operators A : E  E* . However, for Lipschitz continuous monotone operators, algorithm (5) generally does not converge. Numerous modifications of the extragradient algorithm [ 6, 9–16,24 ] can be used for such conditions.

In this paper, we focus on three algorithms: the Tseng method [ 14 ], the extrapolation from the past method [ 18 ] and more recent forward-reflected-backward algorithm [ 20 ]. We consider their natural (2) (3) (4) (5)

We will denote set of solutions of (3) as S d . Note that set S d is convex and closed. Inequality (3) is sometimes called weak or dual formulation of (2) (or Minty inequality), and solutions of (3) are called weak solutions for (2). For monotone operator A we always have S  S d . In our setting we have S d  S [ 1 ].

Variational inequality (2) can be formulated as a fixed-point problem [ 23 ]: with   0 . Formulation (4) is useful, as it leads to obvious algorithmic idea. Procedure x  C J 1  Jx   Ax , xn1  C J 1  Jxn   Axn  modifications for problems in Banach spaces using Alber generalized projection instead of the metric projection.

The goal is to estimate the number of iterations of algorithms necessary to obtain an approximate solution of a given quality. The quality of approximate solution x  C of variational inequality (2) will be measured using the non-negative gap function [ 6 ] versa, if for some x  C we have Gap x  0 , then x is a solution of (2).

Everywhere below we assume, that set C  E is bounded.

4. Algorithms

Let us study the next iterative extragradient-type algorithms for finding solutions of variational inequality problem (2).

Algorithm 1. Modified Tseng method ([ 24 ]).

Select x1  E , n  0 . Set n  1 . 1. Calculate

If yn  xn , then STOP. Else calculate yn  C J 1  Jxn  n Axn  .

xn1  J 1  Jyn n  Ayn  Axn  . 3. Set n : n  1 and go to step 1.

Algorithm 1 is a modification of forward-backward-forward method by P. Tseng [ 14 ] for problems in Banach spaces, where generalized Alber projection is used instead of the metric one.

The weak convergence of the Algorithm 1 in 2-uniformly convex and uniformly smooth Banach space is proved in [ 24 ].

Note that in the case of a Hilbert space and without constraints, the Algorithm 1 coincides with the Korpelevich extragradient method.

Algorithm 2. Extrapolation from the Past.

Select x1  y0  E , n  0 . Set n  1 . 1. Calculate

2. Calculate

yn  C J 1  Jxn  n Ayn1  .

xn1  C J 1  Jxn  n Ayn , if xn1  yn  xn , then STOP. Else set n : n  1 and go to step 1.

Algorithm 2 is a modification of L.D. Popov algorithm [ 17 ] for problems in Banach spaces using generalized Alber projection operator instead of the metric one.

The convergence of Algorithm 2 in a Hilbert space and in Euclidean space with Bregman divergence instead of Euclidean distance is proved in [ 18, 19 ].

Algorithm 3. Operator extrapolation.

Select x0  x1  E , n  0 . Set n  1 . 1. Calculate xn1   J 1  Jxn n Axn n1  Axn  Axn1  .

C 2. If xn1  xn  xn1 , then STOP. Else set n : n  1 and go to step 1.

Algorithm 3 is a modification of modern "forward-reflected-backward algorithm" [ 20 ] for variational inequalities in Banach spaces.

Remark 2. Some text Algorithm 3 can be represented in form, which is similar to Algorithm 2:

 yn  C J 1  Jxn  n Axn ,  xn1  C J 1  Jxn  n Ayn .

This formulation indicates conceptual similarity of Algorithms 1 and 3. More precisely, the operator extrapolation algorithm is obtained from Algorithm 1 in the same way, as Algorithm 2 can be obtained from the analogue of the extragradient algorithm

Now let us turn to the analysis of the algorithms, namely, to the estimation of the number of iterations required to obtain an approximate solution of the variational inequality (2) with a given value of the gap function.

5. Analysis

We will prove, that Algorithms 1 – 3, mentioned above, require O  LD  iterations to obtain feasible point x  C , for which Gap x   , where   0 and D  supa,bC d a,b   .

Proof. For arbitrary y C we have d  y, xn1   d  y, J 1  Jyn  n  Ayn  Axn    y 2  2 Jyn n  Ayn  Axn , y  Jyn n  Ayn  Axn  *2   y 2  2 Jyn , y  2n Ayn  Axn , y  Jyn n  Ayn  Axn  *2  2  d  y, yn   yn 2  2n Ayn  Axn , y  Jyn n  Ayn  Axn  * . d  y, xn1   d  y, yn   2n Ayn  Axn , yn  y n2 Ayn  Axn *2 . (7) d  y, yn   d  y, xn   d  xn , yn   2 Jxn  Jyn , y  xn .

Using Lemma 2, we get an estimation

Let’s use 3-point identity to transform d  y, yn  and now use this in (7)

d  y, xn1   d  y, xn   d  xn , yn   we have

Identity is equivalent to inequality

2 Jxn  Jyn , y  xn  2n Ayn  Axn , yn  y n2 Ayn  Axn *2 .

From Lipschitz continuity of A , equality d  xn , yn   2 Jyn  Jxn , yn  xn  d  yn , xn  and Lemma 1 d  y, xn1   d  y, xn   1n2L2  d  yn , xn  

2 Jxn  Jyn  n Axn  n Ayn , yn  y .

yn  C J 1  Jxn  n Axn  Jxn  n Axn  Jyn , yn  y  0 . which was required to prove. ■

Corollary 1. Let xn,  yn  are the sequences, generated by Algorithm 1 with n  2 1L . Then for the sequence of means zN  N1 nN1yn the next inequality holds:

GapzN   L  supyC dy,x1 .

N

For algorithm 2 we have the next result.

Theorem 2. Let xn,  yn  are the sequences, generated by Algorithm 2. Let n 0, 2L1 . Then

dy,xn1 dy,xn1n2L2dyn,xn 2n Ayn, yn  y .

From monotonicity of A operator it follows

dy,xn1 dy,xn1n2L2dyn,xn 2n Ay, yn  y .

Let’s transform (8) as

2n Ay, yn  y  dy,xndy,xn11n2L2dyn,xn.

Summing (9) over n from 1 to N , we get

N 2n1n Ay, yn  y  dy,x1, which leads to where zN  nN1nyn . Passing in (10) to the supremum over yC , we obtain nN1n

Ay,zN  y 

1 2nN1n

dy,x1 , 1 2nN1n GapzN   supyC dy,x1, (8) (9) (10) for the Cesaro means sequence zN  nN1nyn the next inequality holds: nN1n GapzN  supyC dy,x11L dx1, y0 .

2nN1n

Proof. For arbitrary yC we have

d y,xn1  d y,xnd xn1,xn 2n Ayn, y  xn1 .

From monotonicity of A follows

Ayn, y  xn1  Ayn, y  yn  Ayn, yn  xn1  Ay, y  yn  Ayn, yn  xn1 . So, we have dy,xn1 dy,xndxn1,xn 2n Ayn, yn  xn1  2n Ay, y  yn   dy,xndxn1,xn 2n Ayn1,yn  xn1  Let’s write 2n Ayn1, yn  xn1 as From the inclusion xn1C and Lemma 3 follows inequality

As a result, we have

Jxn nAyn1  Jyn,xn1  yn  0. 2n Ayn1, yn  xn1  2 Jxn nAyn1  Jyn,xn1  yn  2 Jyn  Jxn,xn1  yn .

2n Ayn  Ayn1, yn  xn1 2n Ay, y  yn . (11) Estimating the right side of (11) with (12), we come to the next inequality: dy,xn1 dy,xndxn1,yndyn,xn Now let’s estimate term 2n Ayn  Ayn1,yn  xn1 . We get 2n Ayn  Ayn1,yn  xn1 2n Ay,y  yn . (13) 2n Ayn  Ayn1,yn  xn1  2n Ayn1  Ayn * xn1  yn  2nL yn1  yn xn1  yn   2nL2 2 yn1  yn 2  12 xn1  yn 2  1 Estimating norms in (14) using inequality from Lemma 1, we get

2n Ayn  Ayn1,yn  xn1 nL dxn,yn1 Using (15) in (13), we get  y,xn1 dy,xn1nL 2dxn1,yn1nL 1 2dyn,xn nL1 2dyn,xnnL 2dxn1,yn. (15) nL dxn,yn1 2n Ay,y  yn . (16)

Let’s rewrite (16) as

2n Ay,yn  y  dy,xnnL dxn,yn1 dy,xn1n1L dxn1,yn 1Ln1  2ndxn1, yn1nL 1 2dyn,xn. (17)

Summing (17) over n from 1 to N , we get

N 2n1n Ay, yn  y  dy,x11L dx1,y0, and so

Ay,zN  y  dy,x11L dx1,y0 , 2nN1n where zN  nN1nyn . Passing in (18) to the supremum over yC , we obtain nN1n

GapzN  supyC dy,x11L dx1,y0 ,

2nN1n GapzN   L 32supyC dy,x1 16 dx1, y0 .

N which was required to prove. ■

Corollary 2. Let xn, yn are sequences, generated by Algorithm 2 with n  31L . Then for the means sequence zN  N1 nN1yn the next inequality holds:

Let’s study Algorithm 3. (18) 152

sequence of Cesaro means zN1  nN1nxn1 the next inequality holds

nN1n GapzN1  Proof. For sequence xn, generated by algorithm 3, the next inequality holds 2 nAxn n1Axn  Axn1,y  xn1  dy,xndxn1,xndy,xn1 yС . (19)

Let’s rewrite (19) the next way:

Summing (20) over n from 1 to N , we get

dy,x1dy,xN1 dy,xndy,xn1 2n Axn1,xn1  y 2n Axn1  Axn,xn1  y  2n1 Axn  Axn1,xn  y  2n1 Axn  Axn1,xn1  xn dxn1,xn . (20)

N  2n Axn1,xn1  y 2N AxN1  AxN,xN1  y  n1

nN1n

N  2n Axn1,xn1  y NL xN1  y 2.

n1 So, we can come to inequality

N 2n Axn1,xn1  y NL xN1  y 2 dy,xN1dy,x1 yС . (22) n1 Using monotonicity of A, we get

N N  N  n Axn1,xn1  y  n Ay,xn1  y n  Ay,zN1  y , (23) n1 n1  n1  where zN1  nN1nxn1 . Using estimation (22) in (23), we come to inequality  N   2n  Ay,zN1  y  1 NL xN1  y 2  d y,x1 yС ,  n1   from which follows  21 xN1  xN 2.  N  Gap  zN 1   sup Ay, zN 1  y   2n  sup d  y, x1  , yC  n1  yC 1 which was required to prove. ■ sequence zN 1  N1 nN1 xn1 the next inequality holds

Corollary 3. Let  xn  be a sequence, generated by Algorithm 3 with n  Gap  zN 1  

L

sup d  y, x1  .

N yC

1 2 L . Then for the means

6. Conclusions

Three new extragradient-type algorithms for solving monotone variational inequalities in a Banach space are studied in the paper.

The first two algorithms are natural modifications of Tseng's method [ 14 ] and “Extrapolation from the Past” method [ 18 ] for problems in Banach spaces using generalized Alber projection. Iteration of each of these algorithms is more economical than iteration of the extragradient method. The first algorithm has less projections, the second has less operator calculations.

The third algorithm, called the method of operator extrapolation, is a variant of the Malitsky–Tam forward-reflected-backward algorithm [ 20 ]. Generalized Alber projection is used instead of the metric projection onto feasible set. An attractive feature of the algorithm is only one computation of the operator value and of the generalized projection onto the feasible set at the iterative step. For variational inequalities with monotone Lipschitz operators acting in a 2-uniformly convex and uniformly smooth Banach space, O  1  complexity bounds are proved in terms of the gap function.

Let us point out two actual questions, related to the current research. First, fast and stable algorithms for computing generalized Alber projection for a wide range of sets are needed to efficiently apply algorithms to nonlinear problems in Banach spaces. Second, all results were obtained for the class of 2-uniformly convex and uniformly smooth Banach spaces, which does not contain spaces Lp and W m p ( 2  p   ), which are important for applications. It is highly desirable to get rid of this limitation.

Acknowledgements

This work was supported by the Ministry of Education and Science of Ukraine (project “Mathematical modeling and optimization of dynamic systems for defense, medicine and ecology”, state registration number 0119U100337) and the National Academy of Sciences of Ukraine (project “New methods of research of correctness and solution search for discrete optimization problems, variational inequalities and their applications”, state registration number 0119U101608).

References

[1] D. Kinderlehrer , G. Stampacchia, An Introduction to Variational Inequalities and Their Applications , Society for Industrial and Applied Mathematics , Philadelphia, 2000 .

[2] A. Nagurney , Network economics: A variational inequality approach , Kluwer Academic Publishers, Dordrecht, 1999 .

[3] Y. Alber , I. Ryazantseva , Nonlinear Ill Posed Problems of Monotone Type, Springer, Dordrecht, 2006 .

[4] F. Facchinei , J. S. Pang , Finite-Dimensional Variational Inequalities and Complementarity Problems, Springer Series in Operations Research, vol. I, Springer, New York, 2003 .

[5] S. I. Lyashko , D. A. Klyushin , D. A. Nomirovsky , V. V. Semenov , Identification of age-structured contamination sources in ground water , in: R. Boucekkine , N. Hritonenko , Y. Yatsenko , Y . (Eds.), Optimal Control of Age-Structured Populations in Economy, Demography, and the Environment , Routledge, London-New York, 2013 , pp. 277 - 292 .

[6] A. Nemirovski , Prox-method with rate of convergence O(1/T) for variational inequalities with Lipschitz continuous monotone operators and smooth convex-concave saddle point problems , SIAM J. Optim . 15 ( 2004 ) 229 - 251 . doi: 10 .1137/S1052623403425629.

[7] G. Gidel , H. Berard , P. Vincent , S. Lacoste-Julien , A Variational Inequality Perspective on Generative Adversarial Networks , arXiv preprint arXiv: 1802 . 10551 . ( 2018 ).

[8] M. Liu , W. Zhang , Y. Mroueh , X. Cui , J. Ross , T. Yang , P. Das , A decentralized parallel algorithm for training generative adversarial nets , arXiv preprint arXiv: 1910 . 12999 . ( 2020 ).

[9] G. M. Korpelevich , An extragradient method for finding saddle points and for other problems , Matecon. 12 ( 1976 ) 747 - 756 .

[10] Yu . Nesterov, Dual extrapolation and its applications to solving variational inequalities and related problems , Mathematical Programming 109 ( 2007 ), 319 - 344 .

[11] F. S. Stonyakin , E. A. Vorontsova , M. S. Alkousa , New Version of Mirror Prox for Variational Inequalities with Adaptation to Inexactness , in: Jacimovic M., Khachay M. , Malkova V. , Posypkin M . (Eds.), Optimization and Applications , OPTIMA 2019 , volume 1145 of Communications in Computer and Information Science, Springer, Cham, 2020 , 427 - 442 . doi: 10 .1007/978-3- 030 -38603-0_ 31 .

[12] F. Bach , K. Y. Levy , A Universal Algorithm for Variational Inequalities Adaptive to Smoothness and Noise , arXiv preprint arXiv: 1902 . 01637 . ( 2019 ).

[13] S. V. Denisov , D. A. Nomirovskii , B. V. Rublyov , V. V. Semenov , Convergence of Extragradient Algorithm with Monotone Step Size Strategy for Variational Inequalities and Operator Equations , Journal of Automation and Information Sciences . 51 ( 2019 ) 12 - 24 . doi: 10 .1615/JAutomatInfScien.v51. i6 . 20 .

[14] P. Tseng , A modified forward-backward splitting method for maximal monotone mappings , SIAM Journal on Control and Optimization 38 ( 2000 ) 431 - 446 .

[15] V. V. Semenov , A Strongly Convergent Splitting Method for Systems of Operator Inclusions with Monotone Operators , Journal of Automation and Information Sciences 46 ( 5 ) ( 2014 ) 45 - 56 . https://doi.org/10.1615/JAutomatInfScien.v46. i5 . 40 .

[16] V. V. Semenov , Modified Extragradient Method with Bregman Divergence for Variational Inequalities , Journal of Automation and Information Sciences 50 ( 8 ) ( 2018 ) 26 - 37 . doi: 10 .1615/JAutomatInfScien.v50. i8 . 30 .

[17] L. D. Popov , A modification of the Arrow-Hurwicz method for search of saddle points , Mathematical notes of the Academy of Sciences of the USSR . 28 ( 1980 ) 845 - 848 .

[18] D. A. Nomirovskii , B. V. Rublyov , V. V. Semenov , Convergence of Two-Stage Method with Bregman Divergence for Solving Variational Inequalities , Cybernetics and Systems Analysis 55 ( 2019 ) 359 - 368 . doi: 10 .1007/s10559-019-00142-7.

[19] L. Chabak , V. Semenov , Y. Vedel , A New Non-Euclidean Proximal Method for Equilibrium Problems , In: O. Chertov et al. (Eds.), Recent Developments in Data Science and Intelligent Analysis of Information , volume 836 of Advances in Intelligent Systems and Computing , Springer, Cham, 2019 , pp. 50 - 58 . doi: 10 .1007/978-3- 319 -97885- 7 _ 6 .

[20] Y. Malitsky , M. K. Tam , A Forward-Backward Splitting Method for Monotone Inclusions Without Cocoercivity , SIAM Journal on Optimization 30 ( 2020 ) 1451 - 1472 . doi: 10 .1137/18M1207260.

[21] E. R. Csetnek , Y. Malitsky , M. K. Tam , Shadow Douglas-Rachford Splitting for Monotone Inclusions , Appl Math Optim. 80 ( 2019 ) 665 - 678 . doi: 10 .1007/s00245-019-09597-8.

[22] H. Iiduka , W. Takahashi, Weak convergence of a projection algorithm for variational inequalities in a Banach space , Journal of Mathematical Analysis and Applications 339 ( 1 ) ( 2008 ) 668 - 679 . https://doi.org/10.1016/j.jmaa. 2007 . 07 .019.

[23] Y. I. Alber , Metric and generalized projection operators in Banach spaces: properties and applications . in: Theory and Applications of Nonlinear Operators of Accretive and Monotone Type , vol. 178 , Dekker , New York, 1996 , pp. 15 - 50 .

[24] Y. Shehu , Single projection algorithm for variational inequalities in Banach spaces with application to contact problem , Acta Math. Sci. 40 ( 2020 ) 1045 - 1063 . doi: 10 .1007/s10473-020-0412-2.

[25] K. Aoyama , F. Kohsaka , Strongly relatively nonexpansive sequences generated by firmly nonexpansive-like mappings , Fixed Point Theory Appl . 95 ( 2014 ). doi: 10 .1186/ 1687 -1812-2014-95.

[26] J. Diestel , Geometry of Banach Spaces, Springer-Verlag, Berlin-Heidelberg, 1975 .

[27] B. Beauzamy , Introduction to Banach Spaces and Their Geometry , North-Holland, Amsterdam, 1985 .

[28] I. Cioranescu , Geometry of Banach Spaces, Duality Mappings and Nonlinear Problems , Kluwer Academic, Dordrecht, 1990 .

[29] H. K. Xu , Inequalities in Banach spaces with applications , Nonlinear Anal . 16 ( 12 ) ( 1991 ) 1127 - 1138 .

[30] Z. B. Xu , G. F. Roach , Characteristic inequalities of uniformly convex and uniformly smooth Banach spaces , Journal of Mathematical Analysis and Applications 157 ( 1 ) ( 1991 ) 189 - 210 .