Introduction

The basis of modern distributed systems are telecommunication networks, which are complex technical systems and usually operate in dynamic environments [1,2]. At the same time, the management of such networks should ensure the solution of tasks that ensure data transmission with a given quality [2]. Given this, telecommunication network management systems should include a subsystem for predicting abnormal situations (overload of data transmission routes, errors, etc.), which will allow the network administrator to take timely preventive measures. Therefore, forecasting the state of telecommunications networks is an important task of network administration.

A lot of research has been devoted to predicting the states of complex technical systems. Among them, the following methods and approaches are most often used. Thus, in [3], a method for predicting computer network states based on biometric algorithms is considered. In [4,5], the method of temporal extrapolation, in [6,7,8], the method of spatial extrapolation, in [9], the method of causal relationship and expert methods, and in [10], a method is proposed in which data on the behavior of an object whose features are related to time are presented as the results of observations at uniform time intervals and are represented by a time series. You can also use the method of paired comparisons, which is considered in the work [11]. In addition, recently, neural network-based approaches have been widely used to predict the states of telecommunication networks and have shown their effectiveness. Such approaches are discussed in [12][13][14][15]. Papers [12,13,14] consider neural networks that allow obtaining the desired results without human intervention with low computational costs, and [15,16] consider hybrid neural networks that allow assessing and predicting the state of computer networks with high accuracy of classification of the current and predicted state of the computer network, and [17] considers the use of a probabilistic neural network to solve the problems of classifying and predicting the state of the network transport environment.

Based on the fact that the forecasting problem is a special case of the regression problem, the following types of neural networks can be used to solve it: multilayer perceptron, radial basis networks, generalized regression networks, Volterra networks, and Elman networks. The analysis [18] of the use of such networks in solving forecasting problems indicates the expediency of using time series computation, which will be based on the Elman neural network.

At the same time, the direct use of this network increases the load on the telecommunications network as a whole, as well as the complexity of computing. This makes it impossible to predict its state in real time. Therefore, the question arises of creating a model that would solve this problem.

The article proposes a modification of the Elman network and a method for predicting the overload of telecommunication network data transmission routes based on it, which allows for effective management of a telecommunication network in conditions of high dynamics and complexity of connections between nodes.

Modified Elman neural network

An Elman neural network is a type of recurrent network. An Elman network consists of a multilayer perceptron with feedback. This function allows to take into account previous actions and accumulate information to support management decision-making based on time series forecasting. In other words, time series forecasting is reduced to the task of interpolation (determining intermediate values of a value) of a function of many variables and solving the problem of approximation (reduction to a simplified form) of a multidimensional function, which inherently affects the quality of forecasting. Figure 1 shows a diagram of the Elman neural network, which consists of three layers: the input (distribution) layer, the hidden layer, and the output (processing) layer. In this case, the hidden layer has a feedback on itself [18,19]. In this diagram, X is the input of the neural network; Y is the output of the neural network; C is the context state for the input X; W is the weight matrix of the input layer; V is the weight matrix of the hidden layer feedback; U is the weight matrix connecting the output of the hidden layer with the input of the output layer; H is the hidden layer of neurons, where each input X is connected to each neuron of the hidden layer; O is the output layer of neurons.

In the Elman network, the forecasting process is simulated by the output signal of some nonlinear dynamic system that depends on a number of factors, including past states of the system. Elman proposed to introduce an additional feedback layer into the network, called the contextual or state layer. This layer receives signals from the output of the hidden layer and, through the delay elements C, feeds them to the previous one, the input layer, thus preserving the processed information from previous cycles within the network [18].

Unlike a conventional feed-forward network, the input image of a recurrent network is not a single vector, but a sequence of input image vectors fed to the input in a given order, with the new state of the hidden layer depending on its previous states. Then the Elman network can be described by the following relations in matrix form:

𝑌 𝑡 = 𝐹(𝑈 × 𝐹(𝑊 × 𝑋 𝑡 + 𝑉 × 𝐶 𝑡 )) (1) 𝐶 𝑡 = 𝐹(𝑊 × 𝑋 𝑡−1 + 𝑉 × 𝐶 𝑡−1 )

(2) where 𝑋 𝑡 is the input signal;

𝑌 𝑡 is the output of the neural network; 𝐶 𝑡 is the context state at iteration t for input X; 𝑊 is the weight matrix of the input layer; 𝑉 is the weight matrix of the hidden layer feedback;

𝑈 is the weight matrix connecting the output of the hidden layer with the input of the output layer;

𝑋 𝑡−1 is the signal at the previous iteration; 𝐶 𝑡−1 is the state of the context at the previous iteration; 𝐹 is the vector of the activation function; H is hidden layer of neurons, where each input X is connected to each neuron of the hidden layer;

O is the output layer of neurons. A telecommunication network can be considered as a set of its elements: information directions, routes, nodes, channels, and service quality characteristics. Therefore, as the input of the neural network, we will have a set of parameters of the network directions in the form of an input signal 𝑋 𝑡 = {𝑥 1 , 𝑥 2 , 𝑥 3 , 𝑥 4 , 𝑥 5 , 𝑥 6 , 𝑥 7 , 𝑥 8 }, where x1 is a type of traffic (voice, video, data) being transmitted; x2 is volume of service traffic between nodes; x3 is an information throughput capacity; x4 is delay of packets in the information direction; x5 is the value of jitter in the information sector; x6 is quality of routes between nodes; x7 is number of packets with errors (IPER); x8 is number of packets lost (IPLR). The input signal is formed as a result of monitoring the elements of the telecommunications network.

The output of the neural network 𝑌 𝑡 is an output neuron (adder) that allows you to calculate the deviation of the values detected by the neurons from the value of the operating state of the telecommunications network routes.

Prediction of telecommunication network route overload using a modified Elman network

The essence of the forecasting is to determine the value of congestion of telecommunication network routes, which is calculated by the parameters of the information direction, in order to meet the requirements of network optimization and quality of service for packets of various types. The architecture of the modified Elman recurrent neural network is shown in Fig. 2.

Figure 2 shows a network with multiple inputs of the Elman network, where the number of neurons in the input layer т and a hidden layer n and q and one output block. in the input layer of neurons to the node j in hidden layer. j v and j q is weights that connect the unit j in hidden layer neurons with a node in the recurrent layer.

The hidden layer looks like this -the inputs of all neurons in the hidden layer are given by the network:

) ( ) ( ) 1 ( ) ( 1 1 1 k s q k c v k x w k NET it l g ij it n j ij it m i ij ji           (3) where ) 1 ( ) (   k u k c jt ji , n i ,..., 2 , 1  , m j ,..., 2 , 1  ; ) 1 ( ) (   k c k q jt ji , n i ,..., 2 , 1  , m j ,..., 2 , 1 

The outputs of hidden neurons are obtained from the expression:

                ) ( ) ( ) ( ) ( 1 1 1 k s q k c v k x w f k u it l g ij it n j ij it m i ij H ji (4)

where the sigmoidal function in the hidden layer is selected as the activation function:

An algorithm for training an Elman network with stochastic time efficiency

Stochastic time efficiency is the minimization of time when training a neural network (in our case, Elman) based on error correction (learning with a teacher). The backpropagation algorithm is a supervised learning algorithm that minimizes the global error using the gradient descent method [20]. For the model of stochastic time efficiency of the Elman network, we assume that the resulting output error

n n т t t е y d   

and sampling error n is defined as:

  2 ) ( 5 , 0 ) ( n n t t n n y d t t E   () ( ) ( ) ( exp 1 ) (     (7)

the effective time function of the data is considered as a function of the time variable. The corresponding error of all the data in each network is then retrained and determined as: The main point of the learning algorithm is to minimize the value of the network route congestion function until it reaches the specified minimum value  by repeated training. At each repetition, the value of the network route congestion function is calculated and the global error is obtained. The gradient of the network route congestion function is defined as

  2 1 1 ) ( 2 1 ) ( 1 n n t t N n n N n n y d t N t E N E          (8)W E E     / .

For nodes in the input layer, the weight gradient ij w is given by the formula:

n n n it jt H n j t ij n ij x NET f t z t E w ) ( ) ( ) (            (9)

for nodes in the recurrent layer, the weight gradient is set by the formulas:

n n n it jt H n j t ij n j c NET f t z v t E v ) ( ) ( ) (           , (10) n n n it jt H n j t ij n j q NET f t q q t E q ) ( ) ( ) (          

, for weight nodes in the hidden layer -a weight gradient j v is given by the formula:

), ( ) ( ) ( n n jt H n j t j n j NET f t z z t E z           (11)

where  is learning speed

), ( Based on this update rule for scales ij w , j v , j q and j z are given by the formulas:

k ij k ij k ij w w w    1 (12) k j k j k j v v v    1 (13) k j k j k j q q q    1 (14) k j k j k j z z z    1(15)

The Elman neural network should change the weights to minimize the error between the network's prediction and the prediction target. Such a procedure can be effectively implemented using the methods of mathematical logic, in particular the method [21].

The Elman network training algorithm includes the following steps [20,22,23]:

Step 1. Normalize the input data. In the Elman neural network, we select 8 parameters as input values in the input layer. Then we define the network parameters, such as the learning rate η, which is between 0 and 1, the maximum number of iterations and the initial weights.

Step 2. First, the scales ij w , j v , j q and j z follow a uniform distribution on the interval (-1, 1).

Step 3. The stochastic efficiency over time is introduced by the function Step 4. Set the minimum error ξ route congestion. Based on the network training goal, the value of the route congestion function is calculated:

     N n n t E N E 1 ) ( / 1 . (16)

If the value E less than the specified minimum error, we go to step 5, if it is greater, we go to step 6.

Step 5. Change the connecting weights: calculate the gradient of the connecting weights , , , , , . Then the weights are changed from the current level to the previous layer

k j j k j j k j j ij ij z z q q v v w w     , ,1 1 1 1 , , ,      k j k j k j k ij z q v w .

When predicting the congestion of data transmission routes in a network, the problem of so-called "dead neurons" may arise. One of the limitations of any competing layer is that some neurons may not be involved. That is, the neurons with initial weight vectors are far removed from the input vectors and never win the competition, regardless of the training period. As a result, such vectors are not used in training and the corresponding neurons never win (are dead). Therefore, in order to enable such neurons to win, the learning algorithm provides for the possibility of a "winning neuron" losing its activity. For this purpose, neuronal activity is recorded based on the calculation of the potential of each neuron in the process of predicting the performance of data transmission routes and neuronal training.

First, the layer neurons are assigned a potential   c p i 1 0  , where c is the number of neurons (clusters). Then:

 if the value of the potential i p becomes less than the level min p , then the neuron is excluded from consideration;  if 0 min  p , then the neuron is considered;

 if 1 min  p

, the neurons win in turn, since in each cycle of searching for a "winning neuron" only one of them is ready to be considered for the possibility of defeating the others. On the kth training cycle, the potential is calculated according to the rule::

            j i p k p j i c k p k p i i i , ) 1 ( , 1 ) 1 ( ) ( min , (17)

where j is the number of the "winning neuron".

After providing equal opportunities for neurons to win and calculating the error, the neuron with the number k will be determined by the formula:

j j k d d min  . (18)

The neurons of this layer are sets that are used according to the above rules (see formula 17). The output value of the layer will be the total potential of all "winning neurons" according to the network direction parameters based on the input values in the input layer.

Step 6. The value of the route congestion function is calculated:

                          m j m i n j g j jt j jt j it ij H j T t s q z c x w f v f y 1 1 1 1 1 . (19)

The learning process ends when this value is equal to the specified minimum value.

Conclusion

A method for predicting the overload of data transmission routes in a telecommunications network based on a modified Elman neural network is presented. The peculiarity of this method is to take into account the characteristics of the network by calculating the potential of the network neurons.

This makes it possible to increase the accuracy and speed of predicting route congestion in the network by increasing the network capacity and reducing the computational complexity of the neural network. The work of the Elman network algorithm with stochastic time efficiency is considered.

The proposed method for predicting the congestion of data transmission routes in a telecommunications network can also be used to predict other computer network states such as data throughput and delay.