In the article, the emphasis is put on the modern arti cial neural network (ANN) structure, which in the literature is known as a deep neural network. A network includes more than one hidden layer and comprises many standard modules with ReLu nonlinear activation function. A learning algorithm includes two standard steps, forward and backward, and its e ectiveness depends on the way the learning error is transported back through all the layers to the rst layer. Taking into account all the dimensionalities of matrixes and the nonlinear characteristics of ReLu activation function, the problem is very challenging. In practice tasks, a neural networks internal layer matrixes with ReLu activations function, include a lot of null value of weight coe cients. This phenomenon has a negative impact on the e ectiveness of the learning algorithm's convergence. Analyzing and describing an ANN structure, one usually nds that the rst parameter is the number of ANNs layers "L". By implementing the hierarchical structure to the learning algorithm, an ANN structure is divided into sub-networks. Every sub-network is responsible for nding the optimal value of its weight coe cients using a local target function to minimize the learning error. The second coordination level of the learning algorithm is responsible for coordinating the local solutions and nding the minimum of the global target function. In each iteration the coordinator has to send coordination parameters into the rst level of subnetworks. By using the input and the teaching vectors, the local procedures are working and nding their weight coe cients. At the same step the feedback error is calculated and sent to the coordinator. The process is being repeated until the minimum of all the target functions is achieved.
A deep neural network is built with topologically and logically uniform modules known as layers. A networks structure includes an input layer, a lot of hidden layers, and nally an output layer. Usually, a network is built with 1 L layers of di erent or identical structure. From a mathematical point of view, a layer could be described by a matrix of weight coe cients W l, an input vector X(l 1), a hidden vector U l, an activation function ReLu(U ) = max(0; U ), and an output vector Xl (Fig.1).
A deep neural network includes L 1 hidden layers and one output layer which contains a di erent activation function. An output layer needs to aggregate a set of partial features from previous layers to achieve the nal result, that is an output signal. Using Fig.1, one can de ne the target function: =
Y )T (XL
Y )
(
For all the hidden layers and for forward calculation, one can write:
U l = W l Xl 1
l = 1 L- number of layers in a deep neural network, U l- the internal vector for layer l, F - the vector of activation function, W l-the matrix of weight coe cients for layer l, Xl 1; Xl - the input and output vector of layer l, accordingly.
The process of selecting an activation function is an essential and di cult task. When building standard networks, usually sigmoid and tanh activation functions are used. A sigmoid function has two areas of value in which the function, in an asymptotic way, achieves the value of zero or one. This characteristic has a negative impact on the derivative value and, at the same time, on the algorithm convergence. At the moment, a new activation function is used in a deep neural network, a Recti ed Linear Unit: ReLu, which is de ned as follows:
f = ReLu(u) = max(0; u) From a mathematical point of view, this function is discontinuous for u = 0. In a computers application, this problem is solved by the accepted value f = 0 for u = 0. 1.1
In computer applications, a back propagation learning algorithm is the most popular one. A
learning error is calculated from an output layer, through all the hidden layers to the input
data. From (
EL = [ 1L; 2L; ::: LNL ]-the vector of an output layer error.
The derivatives of the ReLu function could be written as follows:
(
Finally, formula (
1(U )Lg (XL 1)T
EL 1 = (W L)T f1(U )L ELg (12) The output layer error EL has to be translated into the previous layer L 1, this process will be repeated up to the rst hidden layer. Fig. 3. shows the full scheme for a back propagation algorithm and how a layer's error is translated back through the network. The nal back propagation formulas have a recurrent structure. An algorithm will start from the output layer L and going through all the hidden layers, will achieve the rst hidden layer 1: El 1 = (W l)T
l = 1 : L.
According to Fig. 2. the layer is laid between input vector X(l 1) and output vector Xl. The same border is used for the back propagation error from El to E(l 1). In a standard neural network, a sigmoid activation function is used. An output error is translated back to the rst layer decreasing its value and nally, an algorithm can calculate zero's value. This property has a very negative impact on convergence. Therefor, it is the main reason to use ReLu activation function, especially in a deep neural network. Its derivative is equal to 1 or to 0 - the Heaviside step function. Some derivatives (14) are equal to 0 and the weight coe cient does not change in the actual iteration process (formula 14). The same can be observed in the error back propagation function (formula 13).
Taking into account all the limitations mentioned above, a neural network learning algorithm is decomposed, and a new coordination level is implemented. Two or more levels could be used to improve the learning algorithm convergence.
For a multi-layer ANN, a lot of hidden layers and one output layer are sectioned o . The smaller part will be described as a sub-network. Every sub-network has its own output vector, an input vector of the succeeding one Xl, and a local target function l where l = 1 L . Because of the speci c organization of an ANNs hierarchy there are many sub-networks on the rst level, for each of which local target functions are de ned:
= f 1; 2:::: l::: L 1; Lg (15)
These sets of local tasks have to be coordinated to achieve the global solution. The coordinator, as an independent task, will have its own target function. Taking everything into account, this concept is the base on which one may build the new scheme of the ANN learning algorithm's structure (Fig. 3). It is the neural network 's and the learning algorithm's hierarchical structure. The two-level ANN learning algorithm can be described as a set of procedures. The procedures on the rst level are responsible for solving their local tasks and calculating the part of matrix weight coe cients. The second-level procedure has to coordinate all the local procedures (tasks) using its own local target function. The third-level procedure calculates the learning parameters which are used by the second-level. There is a vertical decomposition and interaction between the procedures. Two types of information are sent between the levels. From the second level to the rst level, one is a downward transmission of control signals:
l - vector of data sent from the coordinator to the two neighboring sub-networks l and l + 1, l = 1 (L 1) - number of sub-networks, From the rst level to the second level two sets of feedback signals are sent: forward feedback errors, which are generated by every sub-network when all sub-networks calculate their local target functions value l:
EF = (E 1F ; E 2F ; :::E FL 1) = ( 1; 2; ::: L 1) (16) (17) (18) backward feedback errors, which are calculated by every sub-network in the back propagation procedure:
EB = (E 1B; E 2B; :::E LB) 2.1
The standard ANN learning algorithm is a non-linear minimization task without constraints. To solve this task, iteration procedures are used. Using the most popular back propagation algorithm, one has to choose a lot of control parameters. The algorithm is time-consuming and its convergence is not fast. Dividing the primary algorithm into the sub-network tasks, the local target functions are simpler and can be used in di erent procedures. Additionally, a new procedure is needed: the coordination procedure. In practice, however, the coordinator does not have the ability to nd all the parameters needed for the rst-level procedures. To solve this problem, a multi-level decision hierarchy is proposed [1]. The problems is solved by the iteration algorithm on both the rst and the second level. One can observe speci c dynamic processes. These processes are non-linear and use a lot of control parameters. During the learning process these parameters are stable and do not change. Practice proves that this solution is not optimal. To control the way learning parameters are changed in the iteration process, an additional level could be used - the adaptation level (Fig. 3). Thus, one can build three levels as a minimum:
The local optimization procedures: the algorithm is de ned directly as a minimization task without constraints.
The coordination procedure: this algorithm could be de ned directly as a minimization of the target function as well. Constraints could exist or not. The adaptation procedure: the task or procedure on this level should specify the value of learning parameters not only for the coordinator level, but also on the rst level. To solve this task, a procedure should achieve dynamic characteristic of the learning process from all the levels.
As a conclusion, one can state that the complexity of the problem increases from the rst level to the next one. The coordination and adaptation procedures need more time to solve their own procedure. 3
The deep neural network with an input layer, a set of hidden layers and an output layer can be used for further considerations. As standard ReLu activation function is used for all the hidden layers. For an output layer both sigmoid or ReLu activation functions could be used. A three part learning algorithm will be considered to decompose the standard learning algorithm's structure into sub-network tasks and taking into account Fig. 3. 3.1
All sub - networks are independent because the coordinator sent an input and an output vector l for all the layers. Sub-networks can calculate all values in a parallel way:
1(W 1; X0; 1) = 1 2 (X1 1)T (X1 1) Where: X0; X1 - input and output vector of the rst layer, accordingly, F - vector of activation function. ReLu = max(0; U 1), U 1 - internal vector.
For the all hidden layers l = 2 : (L 1): l(W 1; l 1; l) = 1 2 (Xl l)T (Xl
l) X1 = F (U 1) U 1 = W 1
X0
Xl = F (U l) U l = W l l 1
Xl 1; Xl - input and output vector of all the hidden layers, accordingly, F - vector of activation function. ReLu = max(0; U l), U 1 - internal vector. (19) (20) (21) (22) (23) (24)
XL - output vector, Y - learning vector P epoch included. L(W L; L 1; Y ) = 1 2 (XL
When all sub-networks have nished the forward calculation process, the next step can begin, that is the backward calculation. The calculated error will be sent to the coordinator. Modi ed formulas from subsection 1.1 are used. All subnetworks with their own target functions can calculate their own backward errors in a parallel way:
For the rst sub-network, only the forward error is calculated,(see Fig.4): From formula(19) partial derivatives for the matrix of weight coe cient W 1 are:
EF1 = X1
1(U 1)g (X0)T Formula (29) can be rewritten in the matrix form:
1(U 1) - derivative of the ReLu activation function ReLu0 = max(0; U 1)0 = 1(U 1). The rst sub-network can calculate a new value of the matrix weight coe cients W 1: W 1(n + 1) = W 1(n)
For all the hidden layer l = 2 : (L
1) and using formula (22), partial derivatives for the matrix of weight coe cient W l and an input vector from the coordinator l 1 are:
l Formula (33) can be rewritten in the matrix form: f1(U )l
l E g
1(U l) - derivative of the ReLu activation function ReLu0 = max(0; U l)0 = 1(U l); l 1 = E B l
E l 1 The derivative for the weight coe cients of matrix W l is: (33) (34) (35) (36) (37) (38) (39) (40) (41) (42) (43) l
1(U l)g ( l 1)T W l(n + 1) = W l(n) Formula (38) can be rewritten i the matrix form: A sub-network can calculate the new value of the matrix weight coe cients:
n - current iteration number.
For the output layer using formula (25) partial derivatives for the matrix of weight coefcient W L and an input vector from the coordinator l 1 are: Formula (42) can be rewritten in the matrix form: f1(U )L
L
E g l 1 = EB
L 1
E
L 1 Applying the same procedure as above, a set of formulas is written:
L
1(U L)g ( L 1)T W L(n + 1) = W L(n) (44) (45) (46) (47) (48) (49) (50)
When all sub-networks have nished calculation of their local target functions, the forward E lF and backward E lB feedback information are sent to the coordinator. At the moment all the sub- networks modi ed of their matrixes of weight coe cient W l for l = 1 : L. 4
In a hierarchical learning algorithm, the coordinator plays the main role. It is now time to decide what kind of coordination principle will be chosen. This principle speci es various strategies for the coordinator and determines the structure of the coordinator. In [1] three methods were introduced in which the interaction could be performed:
Interaction Prediction. The coordination input may involve a prediction of the interface input.
Interaction Decoupling. Each rst-level sub-system is introduced into the solution of its task and can treat the interface input as an additional decision variable to be free. Consequently, sub-systems are completely decoupled.
Interaction Estimation. The coordinator speci es the ranges of interface inputs over which they may vary. For a deep neural network containing L layers, the coordinator prepare l = 1 (L 1) coordination signal l. One of them is treated as input vectors and the other as learning data vectors for local target functions l for l = 1 : (L 1). Every coordinator signal is correlated with two feedback signals E lF and E lB. These signals are calculated by the rst level sub-networks and are sent to coordinator. The coordinator uses its own target function: = 1 i=L 1
X 2 i=1 f( i
E Bi)T ( i
E Bi)g + f( i
E Fi )T ( i
E Fi )g To minimize this function, the gradient method is used: (51) (52) (53) The new value of coordination signal is calculated using the gradient method: [E Bi + EF ]
i i(n + 1) = i(n)
This new coordinator signals value is sent to the rst level sub-networks and the entire process is repeated. 5
In formula (53) the learning parameter could be constant or could change during the iteration process. In the beginning the learning process is very dynamic, a large oscillation could be seen in the rst level of the local target functions. Because parameter has to be small, the iteration process is stable. However, although the minimum is achieved asymptotically, it happens very slowly. To improve this algorithm, the coordinator sends the target function value (n) to the adaptation level. The following simple algorithm could be used: If (n) > (n 1) then = kd , where: kd = 0:95 0:98 - decreasing parameter, If (n) < (n 1) then = ki , where: ki = 1:02 1:05 - increasing parameter. The new is sent to the coordinator and used in the iteration process. To make the entire learning process more stable, is usually averaged by epoch: 1 Np
Np
1 Np p=Np 1
X p=1 (n; p + 1) = (n; p) + (n; i) (54) Where: Np - number of input vector in epoch, p - actual index value. 6
In a life insurance company the underwriting process has been playing the central role in risk control and premium calculation. A deep neural network could be used to help the insurance agents to classify the insurance applicant and calculate the rst version of premium. Therefore, a special short questionnaire was prepared which includes only 10 main questions (Fig.4). All the data were divided into three subsets: - learning set includes 250 records, -veri cation set includes 50 vectors,- testing includes 100 vectors.
Dynamic characteristics of the second subnetwork contain two phases. From start to 4000 iteration, error decrease very fast. In the next phase, the learning process is not optimal.
In Fig. 6. the feedback signals do not have optimal characteristics. Probably the learning coe cient is too hight and the coordinator tries to accelerate the learning process. Result is reverse. The adaptation level should force its own strategy to stabilize the learning process. Future works should focus on the coordinator and the adaptation level strategy. The main question is - how to improve the characteristics of the learning process?