Neural control is a kind of adaptive control when artificial neural networks (NN) are used as building blocks of control systems. Neural networks have a number of unique properties that make them a powerful tool for building control systems: the ability to learn from examples and to summarize data, the ability to adapt to changing properties of the object of control and the environment, the suitability for the synthesis of nonlinear regulators. Over the past 20 years, a large number of neurological methods have been developed, the most popular among them are Model Reference Adaptive Neurocontrol and Adaptive Critics [2].
The method of neural control with a reference model, also known as a "circuit with neurotransmitter and neuro-controller" or "reciprocal distribution in time," was proposed in the early 1990s [1], [3 -5]. This method does not require knowledge of the mathematical model of the control object. Instead, a separate neural network, a neuromuscular, studies the direct dynamics of the control object and then it is used to calculate derivatives when training a neuro-controller. At the same time, the trained neuro-emulators with the lowest mean square error of the simulation of the control object usually chooses from the set of trained neuro-emulators. However, is this criterion best if the neural network is used for further training another neural network, connected sequentially to the first, and not actually for modeling the control object?
The paper presents neurocontroller development for satellite rotation control.
NN was proposed in 1943. McCullock and Pitts as the result of studying the structure and activity of biological neurons.
A typical structure of the automatic control system with the PID-regulator and the NN as an automatic adjustment unit is considered in the work [6]. NN acts as a functional transformation, which for each set of input signals the coefficients for the PID regulator are produced. The most complicated part of the design of an NN-based regulator is the training procedure, which reduces to the identification of unknown NN parameters, such as weighting factors and displacement of neurons. For NN training, the gradient search method uses a minimum criterion function, which depends on the parameters of the neurons. The search process is inertial, at each iteration, the search for all coefficients of the network occurs: first for the output layer, then for the previous and so on to the first.
The length of the learning process is a key issue when using NN methods for PID regulators [7]. In addition, when applying NN, there are difficulties due to the impossibility of predicting regulation errors for incoming actions that were not included in the set of training sequences by determining the structure of neurons in the network, the duration of training, the range and the number of training actions.
The main purpose of NM training is to choose the weighting factors of such a network to ensure consistency between input and output values. The neuron with the input p = {p 1 , p 2 , ..., p r } is shown in Fig. 1. The initial value is equal to the scalar product of the vector W on the input vector p, the bias value b is added to the weighted sum of inputs [8].
Output signal is:
... The choice of NN architecture is to determine the number of layers, the number of neurons in each of the layers, the form of the activation function of each layer, and information about the topological links of the neurons. Single-layer NNs are not suitable for solving complex problems [9], but combining several neurons into one or more layers has great potential. The two-layer NN, which in the first layer contains a b а n Inputs
sigmoidal activation function, and in the second one linear, can be trained to approximate any function with finite number of breakpoints with arbitrary accuracy [9].
The purpose of identification is to determine the operator of the model, which converts the input action of the controlled object to the output value. Different identification methods are possible depending on the various forms of representation of mathematical models in the form of ordinary differential equations, difference equations, convolution equations [10], and others. However, none of the proposed methods is universal.
The paper [11] considers the use of NN as an alternative tool for the identification of dynamic objects. The use of NN is based on the fact that in practice modern electric drives are multi-mass systems with nonlinear links. Relevant linearized models built based on transfer functions, cannot always adequately reflect the state of the electric drive in all modes of its operation. The equivalence of a nonlinear system and its linear approximation will be equal in a limited time interval, and when transitioning the output system from one mode to another, it is expedient to use the linearization method and obtain a new linear system.
The paper [12] proposed the use of recurrent multi-layer N with external inputs NARX.
The training model is given as ( 1) ( ( ),..., ( 1), ( )),..., ( 1)
where у(п) is output vector, u(n) is input vector, п is the discrete time moment, q is power of the system. Such a NN, which has feedback with single delays, allows constructing on its basis a model of a dynamic object of arbitrary complexity. Using this method requires veri-
fication of trained NF for adequacy with the use of new data not included in the training sample. Such NN is associated with the possibility of re-training NN [12].
The Matlab [13] Neural Network Toolbox application suite contains the most popular neurocontrollers (NPCs) with
In [14], a mathematical description of predictive neurorization using MATLAB system tools is presented. In [15], the NARMA-L2 controller is used for automatic control of the vessel on a variable course. When solving the problem of guidance and stabilization of the armament of a light armored machine, the NARMA-L2 neuro regulator is used in the contour of speed. As the authors note, NARMA-L2 acts as a relay regulator, whose output is switched to opposite limits, resulting in significant fluctuations in speed (up to 40% of the maximum). However, these neuro-regulators are not connected with physical model of object.
The purpose of this work is to build model and neuro-controller to control small satellite with default amount of reaction wheels.
The main tasks of the paper are to create:
1. Simple simulator of satellite rotations, controlled by 3 or 4 reaction wheels, placed in different configurations. The simulation model will be configurable and easy to read. 2. An Artificial Intelligence (AI) learning module which will trigger the simulator and learn autonomously from the behavior of the simulated satellite, how to control its rotations. 3. The AI module, after trained for different configurations of wheels, will get commands with desired 3D rotation speeds and control the wheels to achieve the desired rotation.
Simulator is developed using C++ programming language. The satellite simulator is created to solve the next tasks:
To provide physical model of physical object; To provide physical model of satellite with reaction wheels for rotation control; To provide possibility to control satellite using reaction wheels during simulation.
Simulator is divided to the such layers of logical implementations:
Core is a general simulation that grants us encapsulated logic for creating and moving of material object. It also allows us to configure simulation and to log information about all objects in simulation. Satellite simulation extends material object logic with reaction wheels and physical facts (friction, gravity, gyroscope effect etc.). The class diagram is given in Fig. 3.
Point -provides an abstract point for further implementations; MassPoint -point which has mass and movement vector; Object -provides enumeration of points which interact with each other; ReactionWheel -inherited from MassPoint instances, is used for changing rotation speed of satellite by changing its angular momentum; Satellite -inherited from Object, provides simulated Satellite of arbitrary form, which moves and rotates using thrusters(ForcePoint) and reaction wheels; Simulator -provides enumeration of Object instances and configuration of scenario of their behavior.
The sphere in the Fig. 4 is a space, which limits the set of material points of the object. The center of mass is note center of the sphere, because its coordinates depends of coordinates and masses off other points. During of training, the neural network must monitor and remember the dependence of the control signal u(k-1) on the next value of the reaction of the control object that was before in the state X(k-1). The values of the control signals and responses of the object are recorded and, on this basis, a training sample is formed.
We used and desired reaction.
In the training mode neural network must find and remember the dependence of control signal ( 1) u k , in state before ( 1) S k . When the object is controlled, the inverse neuro-emulator is connected as a controller and it is receiving the ( ) rr k value from input ( 1) r k :
The class diagram is given in Fig. 5.
Inputs in the control network is the satellite state (speed for each axes). The output is the control signal (torque) u(t). This is energy level for each rotation wheel. We used mini-batch gradient descent algorithm for neural network training.
The neural network structure for this task looks like this: Input layer -3 neurons (for speed by x,y,z), Hidden layer -15 full-connected neurons with sigmoid activation function, Output layer -n neuron with predicted energy level, where n is equal amount of rotation wheels, The bias is used too.
The architecture of neuro-controller is chosen experimentally and given in Fig. 6. The goal of the algorithm is to find model parameters (e.g. coefficients or weights) that minimize the error of the model on the training dataset. It does this by making changes to the model that move it along a gradient or slope of errors down toward a minimum error value. This gives the algorithm its name of "gradient descent."
Mini-batch gradient descent is a trade-off between stochastic gradient descent and batch gradient descent. In mini-batch gradient descent, the cost function (and therefore gradient) is averaged over a small number of samples, from around 10-500. This is opposed to the SGD batch size of 1 sample, and the BGD size of all the training samples.
Mini-batch gradient descent finally takes the best of both worlds and performs an update for every mini-batch of n training examples: θ=θ−η⋅∇θJ(θ;x(i:i+n);y(i:i+n)).
(
This allows us ─ to reduces the variance of the parameter updates, which can lead to more stable convergence; ─ can make use of highly optimized matrix optimizations common to state-of-the-art deep learning libraries that make computing the gradient w.r.t. a mini-batch very efficient. Common mini-batch sizes range between 50 and 256, but can vary for different applications.
For neuro-controller realization we used 1. Eigen to provide vectors, matrixes, quaternions of different dimensions and working with them (it was mostly used in simulator) [16]. 2. MiniDnn to provide neural network for creating controller of a satellite. Parameters of NN is saved in NeuralConfig.h. These neural network parameters were chosen experimentally. We provided more than 500 training experiments with different neural network configuration. In the best attempts mean loss was be equal 0.013 and parameters there was:
To sum up this article described how we could use neural networks for controlling satellites. Neural controllers is a very powerful method that allows us automate different processes and improve accuracy of its results.
An experimental study of the proposed criterion of 500 neuro-controllers was conducted, which showed its effectiveness compared to the traditional method (Loss function value is less than 0.05) of selecting neurotransmitters based on the least square root error method on the test data voter.
In the framework of further research, it is planned to test this criterion, along with other methods of neuro-control, which include the stage of preliminary neuroidentification of the control object: predictive model neuro management and hybrid neuro-PID control as well as using the Kalman cube filter.