The article to the development of mathematical models of complex dynamic objects (using the example of helicopter turboshaft engines) in the form of recurrent neural networks and their use in complex modelling to identify the parameters of automatic control, monitoring, and diagnostic systems is described. For the first time, a concept has been created for constructing neural network models of complex dynamic objects, in which, by increasing the robustness of the functioning of a trained neural network, it becomes possible at its output to increase the reliability of solving problems of identifying complex dynamic objects. The use of a hybrid neural network NARX with a radial-basis nonlinear layer is proposed. The introduction of a radial basis nonlinear layer into the NARX neural network is an effective addition when working with unstructured data such as images, audio signals, or text due to its ability to extract and represent complex patterns and features in these data. This is confirmed by the results of modelling the losses of the neural network, which turned out to be stable over 500 epochs of its training and did not exceed 0.025 (2.5 %). A comprehensive modification of the Levenberg-Marquardt training method is proposed, which consists of a particular application of the Broyden method for calculating the elements of the Hessian matrix, as well as an analytical description of the regularization parameter through the use of control coefficients for increasing or decreasing its value in the event of a neural network training error. The use of the modified Levenberg-Marquardt method made it possible to reduce the average training error of the NARX hybrid neural network with a radial basis layer by 33 % to the level of 0.025.
Experimental research and modelling of complex dynamic objects, for example, helicopter
turboshaft engines (TE) and their control systems, are constant elements of knowledge of their
behaviour throughout the entire life cycle. This starts from the design stage, fine-tuning, and
certification, and ending with operation and disposal. Such studies require a special integrated
modelling technology creation. It makes it possible to confirm the reliability, operability and
required characteristics of systems both before putting them into operation and in operating
modes [
The technology of semi-natural modelling of complex control objects has been used for a long time
in many industries, where real systems are coupled with mathematical models of control objects.
However, the methods and tools for creating such models often remain the same [
One of the most pressing and important problems is to ensure the adequacy of the model of a
complex dynamic object in the automatic control, monitoring, and diagnostics system. The
operation of control, monitoring, and analysis algorithms at the same time can cause various
collisions that need to be taken into account and modelled when developing and configuring a
control system [
The widespread application of intelligent technologies utilizing neural networks has been
observed in the research and development of sophisticated control and monitoring systems
tailored for complex dynamic objects [
The research aim is to increase complex modelling and testing efficiency of a real automatic control system (ACS) for complex dynamic objects, monitoring, and diagnostics through the use of nonlinear dynamic mathematical models and their systems in the form of neural networks (using the example of helicopters TE). A scientific concept is proposed for the neural network model constructing for a complex dynamic object (using the example of helicopters TE), including algorithms for training and identifying a mathematical model of the engine using real data with a choice of the structure and size of the neural network.
Currently, neural networks are an effective means of solving a wide range of problems in complex dynamic objects identifying and their control systems [16, 17]. They are distinguished by the simplicity of the architecture and high representative power. The quality of operation of these networks largely depends on the efficiency of data clustering, as a result of which the centres of activation functions and their dispersion are determined. The works of individual domestic and foreign scientific schools are devoted to the issue of automating the neural network architecture selection. Also, this work focuses both on local modifications of training algorithms [18, 19] and on the use of bionic models [20, 21] to optimize the number of neurons in the hidden layer [22, 23]. The latter has good potential. Because it leads to growing interest in the use of distributed intelligent systems to optimize neural network architecture [24, 25]. An alternative to the described solutions is methods based on special approaches to density clustering. As a result, in hidden layer the optimal neurons number is determined and their key characteristics are established [26, 27]. A common disadvantage of most known solutions is the requirement for the completeness of processed samples. This makes the use of such methods for working in systems with dynamically changing data, for example in control systems, ineffective compared to such specialized neuroarchitectures as Jordan networks [28, 29], Elman networks [30, 31] or recurrent multilayer perceptron [32, 33].
To solve the above task of complex dynamic objects and their control systems, an intelligent system (Fig. 1) can be used that implements the Fault Detection and Identification Method (FDI) [34]. It is based on a neural network mathematical model of the research object and an identification block [35, 36]. Such a system makes it possible to detect and classify abnormal operating modes of the research object, measuring channels and actuators under operating conditions. The output parameters of the mathematical model can be used to diagnose abnormal operating conditions of the research object based on a comparison of the matching the computed parameters with the observed ones. Additionally, they used and also as to restore lost data of U
∆ 2( ) measuring channels in the event of their failure being detected. Such a model should have several special properties, the most important of which are [37, 38]:
1. The model must describe the properties of the research object that determine the nonstationary nature of work processes. This means the need to use a dynamic model.
2. The structure of the mathematical model of the research object should provide the practical possibility of its functioning in combination with mathematical models of other elements of the system.
Neural network model Actuators
Complex dynamic objects
Sensors
Ym Y ε Identification block
The mathematical representation of the nonlinear dynamic model of complex dynamic objects, taking [39] into account, can be represented as a differential equations system: ∆ 1( ) = 11 ∙ 1( )+ ⋯ + 1 ∙ −1( )+ 11 ∙ 1( )+ ⋯ + 1 ∙ ( ), = 21 ∙ 1( )+ ⋯ + 2 ∙ −1( )+ 21 ∙ 1( )+ ⋯ + 2 ∙ ( ), ⋯ (1) = 1 ∙ 1( )+ ⋯ + ∙ −1( )+ 1 ∙ 1( )+ ⋯ + {
Currently, neural network models built on linear neural networks, for example, multilayer perceptron, have been acquired for the identification of complex dynamic objects [40, 41]. Neural network models based on linear neural networks (as multi-layer perceptron) may face several disadvantages when modelling complex dynamic objects. First, they are limited in their ability to capture complex nonlinear relations between variables. This can lead to insufficient accuracy in predicting or modelling. In addition, multi-layer perceptron can suffer from the problem of gradient damping when training deep models. This makes training difficult and can lead to low performance in practice. Also, they can be prone to overtraining in the presence of a limited amount of training data. This makes them less effective for processing real dynamic systems. Therefore, it is expedient to use dynamic recurrent neural networks, for example, recurrent multilayer perceptron (NARX). The justification for the transition from a multilayer perceptron to a recurrent multilayer perceptron (NARX) is based on several factors and is given in Table 1, and the scientific features of this transition are in Table 2. ∙ ( ).
In [40], the use is justified by a modified version of a recurrent multilayer perceptron (NARX). It is a dynamic network characterized by the delay of output/input signals combined in a network input vector, with a radial basis nonlinear and a linear recurrent layer. It should be noted that [40] uses a Gaussian NARX framework with input data regressor selection using a modified gradient method from [41]. This modification is justified based on the outdated NARX models with outdated machine training models [42]. The modified NARX structure proposed in [40] consists of two parts: nonlinear and linear blocks (Fig. 2, where σi is i-th element radial function width; 1, 2, . . ., are i-th element coordinates centre; 1, 2, . . ., are the input signals).
Such a model in a neural network form with feedback makes it possible to take into account the nonlinear dynamic characteristics of an object and guarantee the structural and parametric adequacy of its analytical model. The vector u placed to the input has the form u(t) = [1, u(t), u(t – 1), … u(t – Nu), y(t – 1), ..., u(t – Ny)]T, where Nu – is the input signal delays number, Ny – is the output signal delays number [40]. Depending on the complex dynamic object model, the vector u is formed according to the parameters specified in the technical specifications.
According to [33, 34, 37], the network output vector has the following mapping [40]: ( + 1)= ( ( ), ( − 1), … , ( − ), ( − 1)… , ( − )), (2) then the NARX hybrid network is characterized by a set of numbers (Nu, Ny, Ni), where Ni is neurons in the i-th hidden layer number.
Output Inputs u: h, TN, PN, ρ, nTC, nFT, TG
y(t) u(t – u(t –
... u(t – Nu) y(t – y(t –
... y(t – Ny)
2 i2 n ( u j − cij )2 − j=1
2 i2 n ( u j − cij )2 − j=1 2 i2 ... n ( u j − cij )2 − j=1
2 i2
y
To the aforementioned, the comprehensive schematic representation of configuring the neural network model parameters for complex dynamic objects (using the example of helicopters TE [40]) is presented in Fig. 3, where ; Δwij is the neural network synaptic connections increase; Y = (y1, y2 ..., ym)T is the object output parameters vector; U = (u1, u2 ..., um)T is the input influences vector; = ( 1 , 2 , … , ) is the neural network outputs vector [40].
Modifications of training of neural networks are conducted to improve performance or adjust the model for the specific requirements of the task (Fig. 3). Modifying a neural network training includes changing its structure based on adding or removing layers and neurons, changing activation functions, and adjusting hyperparameters. Also, this includes training rate or regularization parameters, and introducing additional training methods, such as data augmentation or based on pre-trained models for transferring training. Modification may also be necessary if the input data characteristics change, the performance requirements change, or the problem that the neural network needs to solve changes. Control influences vector conversion into initial parameters vector is elucidated by operator F [40]:
The identifying helicopter ЕУ task using a neural network can be formulated: using the results of the proposed training process for a neural network. It forms vectors (Ui; Yi) training set obtained experimentally for a separate engine instance. Aim is to find operator FNN within neural network architectures class. The F operator approximation by FNN operator is deemed optimal if a specified functional from the difference (Y – YNN) does not surpass a given small value εadd, defining the F operator approximation accuracy [40]: = ‖ − ‖ = ∑ 2 ≤
. = ( ).
(3) (4)
The condition (4) satisfaction is guaranteed by neural network training. It involves finetuning parameters using the training sample {(U, Y)} and is verified on a meticulously organized test sample [40]. A scientific concept of direct neural network model construction based on complex dynamic objects is proposed, which is shown in Table 3. (modelling and testing algorithms, as well as the choice of software or hardware implementation) emphasize the practical suitability of the concept for implementation in a real intelligent system for the identification of complex dynamic objects.
For the practical implementation of the proposed scientific concept of the step-by-step creation of a neural network model of complex dynamic objects (Table 3), attention should be drawn to the indicator of model robustness (generalization ability). This is the stability of modelling results to input data disturbances. Evaluation of the resilience or generalization capability of a neural network model is conducted using an algorithm grounded on incremental multi-criteria training [44].
Complex dynamic object model training using a neural network (Fig. 2) is performed by sequentially presenting pre-prepared delay vectors and corresponding output values while simultaneously adjusting the weights of hidden layers by a certain procedure [45, 46]. The neural network training process includes the application of the Levenberg-Marquardt method. The method combines the steepest descent methods (i.e., minimizing the training error along a gradient) and Newton's method (i.e., using a quadratic model to accelerate the quest for the minimum of the error function [47]. The Levenberg-Marquardt method is intended for optimizing it uses the model mean square error (MSE) on the set training, which it minimizes [40]. nonlinear regression models of the form ( )= ∑ =1( ( )− )2. As an optimization criterion,
The Levenberg-Marquardt method combines the ideas of the Gauss-Newton method and gradient descent. At each iteration, this method updates the parameters u in this way: +1 = − ( + )−1 ∆ , (5) , describing partial derivatives of a model concerning =1 =1 =1 where J – is the Jacobian matrix of
= parameters; Δy – is the vector representing the disparity between the present model values and the real data; I – is the identity matrix; λk – is the regularization parameter that controls the step size at each iteration.
According to the research of Serhii Parkhomenko [47, 48], most often, to search for the Jacobian matrix, various variations of difference formulas for calculating derivatives are used, including the central difference derivative [49]. According to [47, 48] for the hybrid NARX network yi values can be written as: • • take very large values; ( , )= 1 (∑ ̃ ∙ 2 (∑ ∙ + ) ∙ 3 (∑ ∙ + ) + ̃), (6) where y(u, w) shows the dependence of an output value yi on input parameters vector values u and corresponding weighting coefficients w, uj – is the value received by the j-th input neuron, wij – is the weight coefficient connecting the j-th input neuron with i-th nonlinear hidden layer neuron, σi – is the bias coefficient for i-th nonlinear hidden layer neuron, ̃ – is the weight coefficient connecting i-th neuron of the nonlinear hidden nonlinear layer with the output neuron, ̃ – is the bias coefficient for the output neuron, f1(•), f2(•), f3(•) – are the activation functions for the output neuron and neurons of hidden nonlinear and linear layers, respectively, Nu – is the number of the input, Ni and Nl – are the hidden nonlinear and linear layers neurons number, respectively. The radial basis function is chosen as the activation function for neurons of the output and nonlinear hidden layers according to [40] and for neurons of the linear hidden layer – a linear function.
H = JTJ, which is derived from the premises:
According to [47, 48] when calculating the Hessian matrix, it is convenient to use the formula the function y(u, w) has a low order of nonlinearity: the second derivatives 2 ( , )do not matrix H is considered in a small neighbourhood of the minimizing vector w, for which the y(u, w) values are close to the desired fi(u), i.e. | ( )− ( , )| ≈ 0.
With an efficient operation of scalar matrix multiplication, the search for H is fast, while the time for calculating the vector of weight changes depends on the variable number wij. Experiments show [47, 48] that it is rarely necessary to solve more than three systems per iteration, which does not greatly affect the execution time of the algorithm. Calculating the Jacobian matrix takes up the bulk of the work of the Levenberg-Marquardt algorithm. So reducing the cost of searching for it speeds up the neural network training. One of these methods is to abandon the calculation of the completely accurate matrix J in favour of its approximate version. For example, the Broyden method calculates Jn+1 using the matrix Jn calculated at step n according to the formula [47, 48, 50]: +1 = + ℎ ∙ ( , +1)− ( , )− ∙ ℎ
, ℎ = +1 − , ℎ
∙ ℎ
From a theoretical point of view, using this approach at each step of the Levenberg-Marquardt algorithm makes sense. However, in practice, the approximation becomes coarser over time. This affects the JTE gradient vector and requires re-calculation of the Jacobian matrix using more accurate methods after an unsuccessful selection of a vector of weight changes δ. Analytical calculation of partial derivatives improves the accuracy of calculations. This allows you to shorten the process by reusing intermediate data and reducing the number of calls to complex functions. Similar to [47, 48], the following substitution was made in this work: = ∑ ∙ + , = ∑ ̃ ∙ 2( )∙ 3( )+ ̃; 1 ≤ ≤ .
Thus, we get: ( , )
( , ) ̃ =1 =1 =1 = ∙ [ ∙ ∙ ], = [ ∙ ′], = ( , )
= [ ∙ ∙ ], 2 2 ∙ [ ∙ ], 2 2 = ∙ [ ∙ ′], ( , ) [ ∙ ′] ∙ [ ∙ ] ̃
1 + [ ∙ ] = ̃ ∙ [ ∙ ′2] ∙ [ ∙ ] (1 + [ ∙ ])2
. = ,
The analytical method of calculating the Jacobian matrix is not applicable in all cases, since for each new neural network model it is necessary to revise the formulas. However, it requires less computation than using central difference derivatives while maintaining accuracy. Distributing calculations of the rows of matrix J between threads allows it to be filled in parallel since the elements are calculated independently. This corresponds to the ribbon pattern. According to [48] τ – is the maximum number of threads running simultaneously. For τ ≪ N, each thread on average processes about rows of matrix J. Row J represents the minimum unit of processing within a parallel block, Ξ – set of numbers of rows J processed by thread t. Since H* = JTJ, then, according to [48], to save memory, you don’t have to store the JT and J matrices separately. For any ∈ [1, ] and ∈ [1, ], the equality
= swapping the indices. To calculate the Hessian matrix element according to [48] is true, which allows you to get elements JT and J by ∗ = ∑ ∙ = ∑ ∙ ; 1 ≤ ≤ ; 1 ≤ ≤ .
=1
=1
(7)
(8)
(9)
(10)
(11)
(12)
(13)
you need to know all the elements of the p-th and q-th columns of J. The simplest way is to wait
for an entire matrix J to be calculated, but this causes a synchronization point where all processes
must wait for the calculation to complete before continuing. Decomposing the matrix H* into a
sum allows us to avoid this [48]:
∗ = ∑ ([ ] )= [
For each calculated row of matrix J in [48], all its elements are multiplied by each other, which leads to obtaining a matrix term: +| – is the element of the matrix [ ]
By combining the matrices of one stream, a cumulative sum matrix is formed in the form: = ∑ [ ]
+, which, when added with matrices from other streams outside the parallel region, results in a finite Hessian matrix H*.
=1 = ∑ ̃( )= ∑ ̃( ), 1 ≤ ≤ .
Thus, combining the calculations of row J and the matrix H within one parallel block was achieved by distributing the calculations of a scalar matrix product over [t]H. Calculation = ̃( )also requires all elements of the p-th column of matrix J [48]:
By decomposing the vector g into cumulative vectors [t]g for each thread, and then decomposing them into term vectors [ ] obtain a similar calculation method used to calculate H*:
+ for all rows J processed within a single thread, we +| = ̃( )= ̃( ), = ∑ [ ]
+ , = ∑ [ ] , ∈ , 1 ≤ ≤ . =1 partial derivatives for weighting coefficients, using pairs of input and expected values, which provides significant memory savings.
In the dynamic approach, to select the regularization parameter λk at each iteration of the Levenberg-Marquardt method, it is proposed to use an algorithm that adaptively adjusts its value depending on the change in error at the current and previous iterations. Let ΔEk be the change in error between the current and previous iterations, that is, ΔEk = Ek – Ek–1, where Ek–1 is error function value at previous iteration k – 1and Ek is the error function value at current iteration k,. Then we can determine the new value of Λk as follows:
∙ , if ∆ > 0, = { ∙ , if ∆ < 0, ,
if ∆ = 0, where α and β – are the coefficients that control the increase or decrease in the value of Λk in the event of a change in error. For example, α > 1, 0 < β < 1. This approach allows the regularization parameter Λk to be adaptively changed depending on the direction of error change at each iteration. This can help speed up the convergence of the method and improve its efficiency.
An efficient choice of the initial value of the regularization parameter λk in the LevenbergMarquardt method can be essential to ensure fast convergence and avoid potential problems such as overfitting or underconvergence. Mathematically, it can be done this way:
The analytical method is based on c analytically estimating the initial value of the regularization parameter. Also, it takes into account the characteristics of the error function and gradient. For example, if the error function has large values, the initial value of λk can be chosen relatively large to ensure the stability of the algorithm. If the error function has small values, the initial value of λk can be chosen relatively small to allow the algorithm to converge
Heuristic method – heuristics or rules of thumb can be used to select the initial value λk. For example, the initial value of λk may be chosen to be a small fixed value based on an assumption of typical parameter scales or error function values.
Optimization method – optimization methods can also be used to select the optimal initial value of λk that minimizes the error function at the initial stage. For example, one can use grid search or optimization methods such as gradient descent to select the initial value of λk in such a way as to minimize the error function. quickly.
Choosing an effective initial value for the regularization parameter λk can significantly impact the Levenberg-Marquardt method performance. So, it is important to pay attention to this and apply appropriate mathematical techniques to ensure the optimal choice.
Since an error function sometimes contains several local minima or has different scales of change, it is advisable to use the multiscale optimization method to adaptively adjust the step in different parameter directions. At each iteration of the optimization algorithm, a base step size η is selected and used to update the parameters based on an error function gradient. For each parameter ui, its characteristic scale of change Δui is calculated, for example, as the standard deviation or scale of a change in the parameter at previous iterations. The step size η is adaptively adjusted for each parameter ui by its characteristic scale of change, allowing us to take into account scale differences between parameters and adapt the step size for each parameter: These parameters are updated using the adapted step size ηi: =
∆
. ( +1) = ( ) − .
This process allows the step size to be adaptively varied in different parameter directions, which can improve the convergence of the optimization algorithm and help avoid getting stuck in local minima or jumps in the error function due to large-scale parameter differences. Taking into account the above, (5) is rewritten as: +1 = − ([ ] + Λ )
∆ + −1 .
Expression (19) is the modified Levenberg-Marquardt method. The Levenberg-Marquardt algorithm adjusts the neural network weights using a quadratic approximation of the error surface. This approximation ensures that the minimum is quickly found, but the risk of hitting a local extremum on the training surface increases.
To conduct a computational experiment, the TV3-117 TE was selected as the research object. The helicopter TE model parameters include atmospheric parameters (ρ – is the air density, PN – is the pressure, TN – is the temperature, and h – is the flight altitude). The helicopter onboard parameters (TG – is the gas temperature in compressor turbine front, nFT – is free turbine rotor speed, nTC – is the gas generator rotor r.p.m) are normalized to absolute values (Table 4) based on
The training set part (author's research [51, 52]) (17) (18) (19) Number 1 2 3 4 5 … 256
nFT
In complex dynamic object identification problems, error surfaces with numerous plateaus and valleys are often encountered, which makes the local minima task one of the main difficulties in achieving maximum efficiency. To overcome this problem, two heuristic approaches were developed and tested to prevent the search process from becoming trapped in local minima [53].
According to [53], the first heuristic requires the algorithm to take risky steps in a random direction along the error surface with increasing step length, to local minimum jump out. After several unsuccessful attempts, in this case, 4, the minimum found is considered the smallest, and the algorithm completes its work by the rules of the original algorithm. This heuristic was tested on 4 data sets, including 64 input numerical variables and 1 output variable. For each data set, the optimal neural network architecture is determined, including the number of neurons in the hidden layer, while minimizing the error. This architecture was identified by an exhaustive search of options, varying from 1 to 20 neurons in the nonlinear layer and from 1 to 10 neurons in the linear layer. After this, similarly [53], 50 test runs were carried out. Each of these included 10 neural network training with different initialization of weights for each data set, without using heuristics. At the final stage, another 50 test runs were performed, within each of which 10 networks were trained with different initialization of weights for each data set, using the heuristics proposed in [53]. The results of testing the neural network are shown in Table 5. Thus, the use of heuristics increases the likelihood of finding successful solutions. However, it is worth noting that in some cases this can lead to a loss of a previously found optimal solution, followed by getting stuck in a local minimum discovered later. This disadvantage can be surmounted by augmenting the random step number, which, in turn, requires additional computing resources [53]. The recommended 4 attempts provide the optimal balance for our subject area. According to [53], the second heuristic is to change the neural network weights when a local minimum is reached, calculating the weighting coefficients using the formula: = + ∙ , (20) where θ1 ≤ θ ≤ θ2 – random number. According to [53], the θ1 and θ2 values are chosen through empirical means as the outcome of an experiment. Various values were tested in the range [–0.5; 0.5] with steps of 0.05 [54]. As a result, the following optimal values were obtained: θ1 = –0.035, θ2 = 0.035.
Thus, when a local minimum is reached, the weights of the neural network undergo random changes by an amount not exceeding θ in both directions. Large values of θ often reset the previous training phase, requiring a neural network to recover and start training again. This, in essence, leads to the fact that the heuristic goes into the mode of a series of tests with different initial weights. Small values of θ may prevent the network from exiting the local minimum. And it will cycle back to an original minimum without benefiting from the heuristic. After shaking the scales, training continues according to the rules of the original algorithm. If the new minimum is not reached, a random change is made again. Through experimentation, it was revealed that the optimal number of random changes should not exceed three. To evaluate the effectiveness of the 2nd heuristic, the same data sets were used as the first. The optimal neural network architectures found at the previous stage were also used. Similarly, 50 test runs were carried out: 10 trainings of neural networks with different initialization of weights for each data set, using the second heuristic. The results of testing the neural network are shown in Table 6. Thus, the second heuristic, like the first, significantly increases the probability of detecting a global minimum. But it has the same drawback as the first – in some instances, it may result in the loss of a previously found optimal solution. In addition, its use requires an empirical selection of boundaries for changing the parameter θ.
Labview created specialized software that uses a modified Levenberg-Marquardt algorithm and automates the recording of experimental results (Fig. 4, 5). The program implements the following single-threaded and parallel options: the traditional Levenberg-Marquardt method with the calculation of J using the central difference derivative formula; the modified LevenbergMarquardt method with the calculation of J by the Broyden method once every two epochs with the two heuristic approaches described above; the traditional Levenberg-Marquardt method with the calculation of J using analytically derived formulas.
For testing, we used a personal computer running the GNU/Linux operating system with an AMD Ryzen 5 5600 processor. It has 6 cores with 12 threads operating at a frequency of 3.3 GHz and 32 GB RAM for DDR-4. The computational experiment aims to assess the execution time of each method, considering the proposed methods for calculating the Jacobian matrix. For training, a NARX hybrid network (Fig. 2) with 7 inputs, in linear layer 5 neurons, in nonlinear layer 20 neurons, and in output layer 1 neuron is used [40]. The instantaneous functional uk+1 (19) is used to assess training according quality to [48]. The calculation results are presented in Table 7. Based on the results of the computational experiment, we can state the following: 1. In hidden layer neurons number increases and the number of steps is limited, the value of E(w) increases and more steps are required to correct the parameters. 2. Using the Broyden method, it was possible to reduce the computation time by approximately 10…12 times compared to the central difference derivative, but uk+1 increased. 3. Direct calculations made it possible to reduce the calculation time by approximately 2.07…2.14 times compared to the central difference derivative. With a small training sample size, direct calculations usually yield the minimum uk+1. 4. Parallel versions of methods work on average about 3…6 times faster than the sequential implementation. 5. uk+1 almost does not change when the number of threads changes, which indicates the correct implementation of parallel processing. The maximum acceleration is approximately 61.38…120.71 times.
The next stage of the computational experiment is devoted to obtaining and analyzing the error in the operation of the trained neural network in the created software product on the identified parameters. Using the training sample (Table 4), identification errors were obtained for the following parameters of the TV3-117 TE: increase degree at compressor pressure (Fig. 6, top left), compressor turbine shaft power (Fig. 6, top right), compressor turbine operation (Fig. 6, bottom left), in combustion chamber fuel consumption (Fig. 6, bottom right), where yellow line is error obtained by the NARX hybrid neural network with the classical Levenberg-Marquardt method [40], red line is error obtained by the NARX hybrid neural network with the modified Levenberg-Marquardt method, green line is approximation line. Analysis of the results of processing the average error of training the hybrid neural network NARX using the modified Levenberg-Marquardt method showed a decrease in the average error value by 33 %, to a level of 0.025.
At the final stage of the computational experiment, the loss was calculated and analyzed in the hybrid neural network NARX (Fig. 7), which serves as an indicator of the variance between the model's predicted values and target variable during training. Loss reflects the degree of error in the model of the research object and is used in the training process to adjust the neural network parameters for minimize this error. The expression for calculating the loss L in a neural network usually depends on the type of problem (such as regression or classification) and the loss function used. For regression problems, the squared error is often used for classification tasks, crossentropy, or other loss functions. The general formula can be represented as the loss sum over all examples of the training set. In this work, used [54, 55]: where n is examples number, yi is actual value of the target variable, ̃ is predicted value by the model, ωi is the weight assigned to each error, allowing their significance to be taken into account in the final loss function. = ∙ ∑ ∙ ( − ̃ )2, (21) blue curve is train; orange curve is validation (author's research)
Fig. 7 shows that the loss function of the hybrid neural network NARX over 500 training epochs is generally stable and does not go beyond the limit of 0.025 (2.5 %), which indicates acceptable losses in problems of identifying complex dynamic objects [56, 57].
Table 8 contains comparative analysis results for thermogas-dynamic parameters identification accuracy in the engine operating process using a neural network and classical methods for each parameter in TV3-117 TE model [40].
We introduced supplementary noise to the dataset to assess neural networks resilience to variations in input information (Table 4). This noise was incorporated into each parameter by integrating white noise with a standard deviation of σi = 0.025 and a mean of zero. For each parameter this corresponds to maximum value2.5 %. Table 9 illustrates the outcomes of a comparative assessment of the precision in implementing the technique for discerning thermogas-dynamic parameters in TV3-117 TE operating process using both neural networks and traditional approaches.
An examination of Table 9 demonstrates that under the stated noise conditions, the error in identification remains within specific limits: for Gaussian NARX model with modified LevenbergMarquardt method is 0.43 %, for Gaussian NARX model is 0.71 %, for three-layer perceptron structured as 7–53–36 is 1.09 % [40, 58, 59], and for thermogas-dynamic TV3-117 TE model is 3.15 %. Due to the maximum absolute error and white noise presence in applying the identification technique for the thermogas-dynamic parameters using the least squares method rose from 1.96 % to 3.15 %. This error increased from 0.64 % to 1.09 % [40, 58, 59] for the threelayer perceptron structured as 7–53–36, , from 0.28 % to 0.43 % for the gaussian NARX model with modified Levenberg-Marquardt method, and it went up from 0.43 % to 0.74 % for Gaussian NARX model. To evaluate the dependability of the neural network approach in discerning the thermogas-dynamic parameters of TV3-117 TE operating process [40, 58, 59], the following formulations can be employed [60, 61]:
Classical Neural network: three-layer perceptron [55, 56] Gaussian NARX-model [40] Gaussian NARX-model with modified Levenberg-Marquardt method where Kerror and Kquality represent coefficients for erroneous and quality identification, respectively [62]; Terror indicates the cumulative time of segments associated with misclassification, while T0 denotes test sample duration (in this context, T0 = 5 s) [63]. both erroneous [40, 60–64], including the relations between the increase degree dependence in compressor turbine operation, compressor turbine shaft power, the total compressor pressure, and fuel consumption in the combustion chamber. Erroneous and qualitative coefficients calculating results (author's research, comparisons with [40])
As depicted in Table 10, the rates of erroneous identification coefficients remain below 0.393 %, while the minimum coefficients for accurate identification rate reach 99.925 %.
The main area of practical application of the developed method is the helicopter TE monitoring and operation controlling neural network on-board expert system [65]. The developed method can be included as a neural network module for helicopter TE parameters identification, which provides continuous and engine operation accurate monitoring in real-time, and also increases the level of safety and flight efficiency.
For the first time, a concept has been created for constructing neural network models of complex dynamic objects, in which, by increasing the robustness of the functioning of a trained neural network, it becomes possible at its output to increase the reliability of solving problems of identifying complex dynamic objects.
The universal neural network model of complex dynamic objects was further developed in the form of a hybrid neural network NARX with a radial-basis layer, in which, through the use of a modified
The transition from linear neural networks (multilayer perceptron) to nonlinear ones (hybrid neural network NARX with a radial basis layer) in identification tasks of complex dynamic objects is scientifically substantiated, providing more accurate and flexible identification of parameters of complex dynamic objects in real-time.
The method of calculating the elements of the Hessian matrix, a component of the analytical expression of the Levenberg-Marquardt method, was further developed, which, by taking into account the weight connections of neurons of both nonlinear and linear layers, allowed for to reduction of the calculation time by approximately 10...12 times compared to the central difference derivative. The use of direct calculations made it possible to reduce the calculation time by approximately 2.07...2.14 times compared to the central difference derivative.
For the first time, an analytical description of the regularization parameter has been proposed, which is based on control coefficients of increasing or decreasing its value in the event of a change in error, in the mathematical expression of the Levenberg-Marquardt method, which significantly affects its performance.
The proposed complex modification of the Levenberg-Marquardt method made it possible to experimentally select the optimal structure of the neural network, reduce the average value of the training error of a NARX hybrid neural network with a radial basis layer by 33 % for the level of 0.025, and also ensure the stability of the loss function of the neural network throughout 500 training epochs, which does not exceed 2.5 %.