The present paper aims to develop a reconstruction method for the right side of a system of ODEs in polynomial form from sparse and irregularly distributed time-series data. This method doesn't require any additional knowledge about the system and has several steps. The scarcity of the data through the trajectory length is compensated by the artificially generated points using approximating trigonometrical polynomials. Then, we get uniformly spread data points with the step conditioned by the desired accuracy of derivatives approximation in ODEs. This let to further use conventional reconstruction algorithms described in the literature. We test the proposed method on time series data generated from known ODE models in a two-dimensional system. We quantify the accuracy of the reconstruction for the system of ODEs as a function of the amount of data used by the method. Further, we solve the reconstructed system of ODEs and compare the solution to the original time series data. The method developed and validated here can now be applied to large data sets for physical and biological systems for which there is no known system of ODEs.
The modeling and identification of dynamical systems from time-series data is a field of increasing interest mainly because of forecasting and model-based control applications. A mathematical description of interconnections between the measured data is a helpful tool to understand the considered dynamical system, predict its behavior or respond to different control actions.
System identification problems can be categorized into two groups: reconstruction of the mathematical form of the measured data (i.e. a solution of the system) or the reconstruction of the equations of motion of the underlying system. Irrelative to the type of problem, when the structure of the model is known the only necessary thing is to estimate the unknown coefficients from the timeseries data either in general solution or in the right side of the differential equation. Otherwise, identification requires a prior step of choosing the most suitable mathematical form. The situation becomes more complicated when the measured data demonstrates nonnegligible nonlinear system dynamics. Unlike linear models for which a complete theory exists, nonlinear modeling still lacks well-established identification algorithms.
The previous work of the authors [1] introduces an algorithm of learning dynamical systems from time-series data including both of mentioned above methods as two sequential steps. At first, reconstruction of ordinary differential equations (ODEs) with a polynomial right side with only one measured system trajectory. Then, applying the Taylor mapping technique to represent the solution of ODEs. The solution is written in a polynomial form establishing a relationship between the phase variables at the current and past moments of time. In other words, a regression formula with predefined weights initialized from the reconstructed ODEs. Such a representation is convenient for further fine-tuning of the weights according to other available training data.
The present paper aims to strengthen the previous research [1] by enhancing a reconstruction algorithm of ODEs with polynomial right side when only rare or irregular measurements are available. Under ideal conditions, i.e. noise-free and high sampling frequency a variety of schemes, including sparse regression schemes [2-3], reservoir computing [4] and neural approaches [5-6]. However, real life data are often corrupted by noise and/or observed partially. In such situations, the abovementioned approaches are most likely to fail to uncover unknown governing equations. To address this challenge, we need to jointly solve the reconstruction of governing equations and the identification of the hidden dynamics [1].
Here, we develop a method to build a polynomial right side of a system of ODEs that will correspond to the time-series data of a dynamical system. This method doesn’t need any input except the time series data and includes several steps. We first identify a basis to approximate the sparse time series data. Here we use trigonometrical bases, but it can be chosen arbitrarily. The second step is an augmentation of a measured system trajectory by a linear segment to satisfy the periodicity condition at the ends of the time interval. Then, a choice both the order of trigonometrical polynomials and data points used for approximation. After the best approximation function consistent with the measured data is found, we use it further for data points generation through the trajectory length and solving a system of linear equations. We test our ODEs reconstruction method on time series data generated from known ODEs models in a two-dimensional system. We quantify the accuracy of the reconstruction for the system of ODEs as a function of the amount of data used by the method. Further, we solve the reconstructed system of ODEs and compare the solution to the original time series data. The method developed and validated here can now be applied to large data sets for physical and biological systems for which there is no known system of ODEs.
The rest of the paper is organized as follows. Sec. 2 introduce a step-by-step description of the proposed reconstruction algorithm, while sec. 3 contains the presentation of a test model with the known system of ODEs and the results of its equation reconstruction in polynomial form. Sec. 4 draws the conclusion remarks and discusses the obtained results.
Let us denote the set of the parameters describing the process as vector with changing in time components ( ), = ̅1̅̅,̅̅. And we suppose to know the values of the vector function ( ) measured in
discrete times 0, … , +1: ( 0), … , ( +1).
The main our assumption about the collected time-series data describing the multi-parametric
dynamical process is that it approximately follows an autonomous ODEs system. We will find its
right-hand side in polynomial form, so that the system looks like
Equation (
⋮ ( ( ) [2]( )
= , … [ ]( 1) … [ ]( 2) , ⋱
⋮ … [ ]( )) = ( 1, … , ) , where is an independent variable, ∈ ℝ is a state vector corresponding to the parameters of the dynamical process, and [ ] means k-th Kroneker’s power of vector . For example, for we have [2] = ( 12, 1 2, 22), [3] = ( 13, 12 2, 1 22, 23) after reduction of the same terms. = ( 1, 2)
Matrices are unknown and should be found from the measurements that we have solving the system of liner equations, that comes in by replacing the derivatives ( 0), … , ( +1). If the available data is of high sampling frequency, we can easily compute in the left side of (1) with finite differences: ( +1)− ( −1) +1− −1 = ∑ =0 [ ]( ), = ̅1̅̅,̅̅̅.
[ ]( ) = ( [ ]( )) ,
) .
(1)
(
+1 − −1
= + 2 + ⋯ + .
The system (
However, in case of sparse measurements in time, this approach can hardly be applied
because system (
To address this challenge, we propose at first to approximate the collected measurement by a set of trigonometrical polynomials. The choice is due to the uniform properties of this set of basis functions. For example, approximating a function by finite sum of Chebyshev polynomials has drawbacks related to the behavior of their derivatives [7]. The algorithm includes the following sequential steps. 1. Periodization of the input dataset. Choosing a closing coefficient ( ).
Let us denote the lattice 0: { 0 ∈ [ 0, . . , +1], = ̅1̅,̅̅̅̅̅} where the data 0: { ( 0)} × 0 is 0 collected. In general case the collected data doesn’t correspond to a periodic or oscillational dynamic process, meaning that 0( 10) ≠ 0( 00 ). To fulfill this condition necessary for trigonometric approximation, we introduce a close coefficient > 1 that enlarge the given time interval till the value
= 0 + ( +1 − 0). This complements the lattice 0 11: { 111, … 1111} , where 111 = 00 and 1111 = . Let 1 = 0 ∪ 11 and lattice function 1 = 0 ∪ 11, where 11: { ( 11)} × 11 = { 11 + } =1̅̅,̅̅̅̅1̅1̅ . Unknown coefficients are defined from with the following values the following conditions: =1̅̅̅,̅̅ { 01 = 11 11
+ 0 0
= 111 +
Satisfying to (
The order of trigonometrical polynomials K defines the number of lattice points 2 = 2 + 1
0
,
,
, = ̅1̅̅,̅̅, = 1̅̅,̅̅̅ calculation in approximation
(
2
where =
, = − 0, - a lattice point where the polynomial value is computed.
Let us choose 2 points from the lattice 1 being at an equal distance from each other and
select corresponding lattice function values 1. Those points form lattice 2: { 12, … , 2 2} and lattice
function 2: { ( 2)} × 2. According to [8], the unknown coefficients in (
Trigonometrical approximation (
Condition (
The algorithm described above has three parameters introduced at each step that make an
impact on the solution of (
is the
As a training dataset for the introduced reconstruction algorithms let us consider points ( ( ), ( )) artificially generated by a model of two-dimensional particle motion in cylindrical deflector:
̇ = , { ̇ = −2 + 2 calculating unknown matrices , = ̅1̅,̅̅̅.
We numerically find a particular solution of (
= 10 and integration step ℎ = 0.01. As a training set
we consider every 20-th point in the generated trajectory, so 0 includes 20 points ( 0 = 20).
approximation (step 1), figure 1,b (blue line) illustrates 4 lattice function (step 2-3) used further for
The final reconstruction results depending on the amount of data used by the method are
presented in figure 2 (a) 0 = 20, b) 0 = 13). The blue line corresponds to the true solution of (
The paper introduces a reconstruction algorithm for the system of ODEs right side in polynomial form from irregularly distributed time-series data. The presented algorithm aims to amplify the results of the authors’ previous work [1] built on the assumption that the collected measurements are frequently and uniformly spread in time. Although the solution of reconstructed ODEs doesn’t fully congruent to the solution of true ODEs [fig. 2], it captures the dynamics of the system what is more important. According to [1, 9], the reconstructed ODEs are only used for initial weights initialization in the regression formula or neural network. After then, the weights are anyway fine-tuned with additional data, so we claim mild requirements for the accuracy of the ODEs reconstruction.
The authors would like to thank Saint Petersburg State University for the research grant ID: 75206008. [1] Golovkina, A. Kozynchenko, V. Kulabukhova, N. Reconstruction and Identification of Dynamical Systems Based on Taylor Maps // Gervasi O. et al. (eds) Computational Science and Its Applications – ICCSA 2021. ICCSA 2021. Lecture Notes in Computer Science, vol. 12956. Springer, 2021