The article discusses the concept of choosing the sequence of control actions in order to minimize the possibility of the system state transition to an adverse one. For this purpose, the bionic model based on the synthesis of information approach, neural networks and a genetic algorithm is developed. The functionality of each of the model elements and their interaction are presented in this paper. Special attention is paid to neuroevolutionary interaction. At the same time, information about control actions is encapsulated in the gene, which allowed increasing the functionality of the algorithm due to multidimensional data representation. The article describes the principle of data representation in bionic models, which differs from the existing ones by the possibility of explicit or implicit representation of the control action in the chromosome. In the explicit representation one neural network is formed, it describes the effect of any of the control actions involved in the training. An implicit view creates a set of models, each of which describes the effect of only one control action. A brief description of the software implemented in the Python programming language is provided.
The development of information technologies for
monitoring, prediction and situation modeling based on
the application of bionic principles is one of the most
promising scientific directions [
By the bionic model we will mean the mathematical model, as well as its software implementation, built on the principle of functioning and organization of biosystems.
In order to implement situation management, namely, to choose the best solution, evaluate and predict the effect of the application of control actions it is necessary to have an adequate mathematical model. This role is performed by the bionic model of choosing the sequence of control actions of the following type:
< , , , > where - neural network models, - genetic algorithms, - information method for calculating the generalized indicator of a biosystem, - model setting algorithms.
Synthesis of information approaches, neural networks and genetic algorithms in bionic models allows systems to exchange information and transfer the values of their characteristics as input to another subsystem which improves the quality of functioning and interpretability.
The application of the information approach in the
structure of the bionic model makes it possible to
evaluate the state of the object of the study at the
considered moments of time and to reveal deviations
from the stationary state. Of interest is the formation of a
single quantitative indicator [
On the basis of implementation and analysis of
methods for construction of generalized estimates in the
information system, the approach [
The choice is justified by the analysis of the results
obtained by:
1. The integral criterion based on the evaluation of
similarity measure of observed and preferred state
areas in feature space, where similarity measure is
normalized in Mahalanobis metric by intramultiple
distance of reference (steady) state area [
Neural networks have been chosen as the basic
technology in the bionic model structure in order to
predict the values of features characterizing the state of
the system in dynamics, as well as when choosing a
particular control action. This is due to the high
efficiency of prediction; the possibility of implementing a
model ensemble and using recursive neural networks for
a variable structure vector [
Let us consider an object whose state is characterized by informative features xi … and the ways to influence the state of the object U = (u1,...,ul), where l is the number of control actions.
It is believed that the value of the i-th feature at a point of time t +1 is determined according to the formula: ( + 1) = ( ( ), ( − 1), … , ( − ), … , ).
The problem comes down to solving the task of predicting the time series. For this purpose, n neural networks are formed, each of which describes the effect of only one control action. Each of the neural networks takes only the vector of indicators of the study object as an input, and the information about the type of the simulated impact is presented in all weights of the model. The output of neural networks is the predicted values of both variables characterizing the state of the research object and generalized indicators obtained on the basis of the information method.
The implementation of the prediction algorithms is
based on recurrent neural networks [
In implicit representation, the result of machine learning is a set of models, each describing the effect of only one control action. Each of the models accepts only the vector of indicators of the study object, and the information about the type of simulated impact is implicitly presented (i.e. distributed) in all weights of the model.
The need to present information in two types is justified by the fact that the explicit representation allows to regulate the "intensity" of the impact, but requires careful presetting of weights and regularization coefficients, which would determine the informativity of the input neurons that take the values describing the control action to the input.
When learning models with implicit representation of control actions, there is no problem of loss of informativity of input neurons values. Each model contains fewer configurable parameters. However, the type of impact and its “intensity” remain unchanged after learning, and a large number of models result in huge investment of time, compared to a model using explicit representation.
The approaches described above are applicable to a wide class of models, from linear regression to dynamic and deep neural networks. Optimization of deep neural network parameters is a computationally more complicated task as "layered" architectures contain combinations of nonlinearities defining a wide range of learning problems described by the general term “deep learning”. 4.
The application of the evolutionary approach in the structure of the bionic model has made it possible both to search for the most effective control actions and to optimize hyperparameters of the models. In this case, control actions are presented in two forms: - in case of explicit representation of control actions, the chromosome is constructed from genes, each of which determines the set of control actions applied to the object of study at some point of time; - in case of implicit representation, the chromosome is determined by genes characterizing certain properties of the model (number of layers, neurons per a layer, short-term memory capacity, degree of network connectivity, type of activation function, type of dynamic network deployment, etc.).
Fig. 3 shows a diagram for the general case where there are several dependent control actions on the test object at each time point. In this case, each chromosome involved in the formation of the population is a matrix (M)ij, where i is the type of action; j is the time point.
Two ways of presenting priori data were reflected both in the construction of the chromosome and in the defining of criterion function.
With explicit representation, a gene is a vector of logical variables, each of which determines the presence or absence of the control action. When the function fitting is calculated, each gene is concatenated with a vector of values of variables describing the state of the study object and it is input to some model. The output value of the model by means of a generalized assessment reflects the effectiveness of situation management. In the implicit setting of control actions, each gene is a vector of neural network model parameters describing a certain action. Thus, some "bank" of learnt models is formed, vectors of weights remain unchanged in the course of the evolutionary process, but they participate in chromosome formation.
Each gene, since it is a fully functional model, takes a vector of variables describing the object of the study as input, and its output is the predicted value of the generalized (integral) assessment of the dynamic system state depending on the control action.
We summarize and note the functionality of each element of the bionic model of choosing the sequence of control actions during situation management.
Let us have a database that stores the values of the properties of the study objects at the following moments of time: prior and after some control action is applied.
1. For each object at each moment of time, we calculate the values of the generalized indicator and evaluate their deviations from the norm.
2. The control action on each object is modeled by a neural network. As input data, values of variables characterizing the state of the research object before applying the control action are used. As the desired responses, both the variable values and the generalized indicator (I) values are used after the control action is applied. The objective function to be minimized is defined as the deviation of the generalized indicator from the norm.
3. The obtained set of neural network models forms a plurality of genes that can be specified by a vector of neural network parameters (for an implicit form of control action) and a vector of logical variables in which the control action is encoded (in an explicit form).
4. The sequence of control actions is determined by known previous states (state variables) and predicted values of object state variables using genetic algorithm. In the process of genetic algorithm work the request to neural networks for obtaining the predicted values of the object state for all future time intervals is performed.
5. The genetic algorithm forms and chooses new generation by means of selection, crossover and mutation.
6. The values of the fitting function of the genetic algorithm are determined as deviations (from the norm) of the value of the generalized indicator predicted by the neural network (by a gene). Among the set of control actions, defined by chromosomes, the one at which the value of the fitting function will be minimal is chosen (the range is set [0,1; 1]).
At the first stage of development of information system in order to test the possibility of using artificial neural networks for predicting the state of dynamic systems (the object of the study) a prototype was developed which implemented the following procedure of actions:
1. Loading of the table with integral criteria on tic marks;
2. Data separation into training and test samples according to cross-validation conditions;
3. Neural network training in order to obtain a predictive generalized assessment of the state of the study object at the next point of time; 4. Calculation of a prediction error; 5. Optimal selection of neural network hyperparameters using a genetic algorithm.
The results of the prototype work made it possible to assess the accuracy of this approach and to conclude on the need to develop a software complex based on the chosen algorithms.
The prototype was developed in Matlab R2008b.
Python, a high-level programming language relating to freely distributed software, has been chosen as the main development tool.
The program on Python consists of a main module and secondary modules that can be connected when running the main module code. Secondary modules are: • NumPy - data interpolation, statistical functions, optimized calculations; • Pandas - dataframe, loading and saving *.csv-files; • PyBrain - artificial neural networks, optimization methods, including genetic algorithm; • PyQt – components of the graphic interface.
The selected tools allow you to move from the stage of prototype development to the implementation of the finished software project quite quickly.
The whole process will be divided into several stages: collection of primary data and secondary data obtained from the results of additional studies. After the calculation of integral indicators based on the information approach, a graph of changes in the state of the research object in time is displayed. The cubic spline interpolates the data and calculates the rate of change of the variables characterizing the state of the research object.
Further, the calculation of predicted values of each of the indicators is performed separately, and a predicted integrated assessment of the state during the situation control period is given. The deviations between the prediction and the stationary (normal) values at each time point are determined and displayed on the graph. A sequence of control actions at which deviation is minimal is formed.
Development and testing of the bionic model, which is based on the interaction of the information method, genetic algorithm and neural networks, allow for situation control based on correct choice of the sequence of control actions. The information method forms a single integral indicator characterizing the state of the object of the study. Artificial neural networks enable to obtain prediction of integral assessment of the object state depending on the chosen control actions. The genetic algorithm chooses a sequence of control actions that reduce the possibility of transition to adverse states.
The principle of data representation in bionic models presented in the article differs from the existing ones by the possibility of explicit or implicit representation of control action in the chromosome. In the explicit representation one neural network is formed, it describes the effect of any of the control actions involved in the training. Information about the type of control action is supplied to the input of neural network. In the implicit representation a set of models is created, each of which describes the effect of only one control action, information about the type of simulated impact is distributed in all weights of the model. With a large number of control actions, training models with implicit representation of the actions is time consuming, in comparison with a model using explicit representation. However, when the information is encapsulated in a gene, the possibility of n-dimensional representation of data appears, which extends the functionality of the algorithm.
The work was supported by RFBR, Grant № 19-0700351.