Stationarity, Ergodicity and Mixing Properties of Conditional Linear Time Series Models

Abstract

We analyze some useful properties of time series which are important in the problems of statistical data analysis and forecasting. The stationarity of time series can be tested in practice, as there are many special tests exist, but the property of ergodicity is usually just assumed to be present. Ergodicity means that the statistical characteristics observed over a single long time series are representative of the statistical characteristics observed across multiple samples of the process at a single point in time. The model-based approach has been used in the paper to justify the stationarity and ergodicity properties of investigated time series. The utilized model is conditional linear time series with known representation of its characteristic function. It has been used for justifying the mixing property which implies ergodicity.

Keywords

• ⋆1
• model
• signal
• conditional linear time series
• stationarity
• ergodicity
• mixing
• characteristic function

1. Introduction

Time series analysis is a statistical technique used to analyze data points collected or recorded at specific time intervals [ 1– 3 ]. The primary goal of time series analysis is to identify patterns, trends, and other characteristics in the data that can be used for forecasting, monitoring, and understanding the underlying processes that generate the data [ 4 ]. It is widely used in various fields, including finance, economics, weather forecasting, engineering, medicine, energy, and environmental science. Very often time series (discrete-time random process) is obtained by sampling or averaging of continuous-time random process in the problems information signals and systems modelling, analysis, and estimation.

0000-0002-8720-6479 (M. Fryz); 0000-0002-6596-3460 (S. Kovtun); 0000-0002-8493-9474 (Y. Kuts); 0000-00026596-3460 (B. Mlynko); 0000-0002-0248-3224 (O. Kuchvara)

© 2023 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).

Many real-world time series are non-stationary, requiring transformation before analysis. But there are enough different methods to test the stationarity (which is invariance of the probabilistic characteristics of the process over time) property using the real data. Theoretical analysis of stationarity property of the time series models can be also performed [ 5 ].

Ergodicity is another useful property of time series, information signals [ 6 ], systems [ 7–9 ], control algorithms [ 10 ]. Ergodicity is a concept in statistics and probability theory that describes the long-term behavior of a system or process. A process is considered ergodic if, over time, its time-averaged properties converge to its ensemble-averaged properties [ 11 ]. This implies that observing the time series over a long period gives you enough information to understand its overall behavior, without needing to observe multiple realizations. This is usually practical since, in many real-world applications, only a single realization of the continuous-time process or time series is available. The ergodicity is important property also for the problems of time series forecasting.

But comparing with stationarity, there are only few tests of ergodicity, based on practical time series analysis. They are related only for some specific classes of random processes, such as Markov processes [ 12 ], [ 13 ], or utilize only mean ergodic property [ 14 ]. That is why, the ergodic property of investigated time series usually just assumed.

But there is another approach, which consists in substantiating the mathematical model of the time series, which is ergodic. For example, continuous-time and discrete-time linear random processes [ 15 ] are ergodic [ 11 ].

The stationarity and ergodicity of the important class of continuous-time conditional linear random processes have been proven in the paper [ 11 ]. We develop the ideas [ 11 ] of utilizing the characteristic functions method in present article proving the ergodicity properties for the class of conditional linear time series (discrete-time conditional linear random processes). The practical importance of such kind of time series for the information signal mathematical modelling, estimation, forecasting and computer simulation have been analyzed in [ 16 ].

The main goal of the article is to prove the mixing condition for conditional linear time series because it implies the ergodicity.

Following the general structure and ideas of the paper [ 11 ], we analyze further the notion of stationary conditional linear time series and its multidimensional characteristic functions. Then we use this tool for proving the mixing and ergodicity consequently.

2. Conditional linear time series and stationarity

We start from conditional linear time series (CLTS) definition and analysis of its characteristic function which is the tool for stationarity, mixing, and ergodicity proving.

A real-valued conditional linear time series ξt ( ω ), t ∈ Z, ω∈ Ω (where Ω is sample space) is defined as a discrete-time conditional linear random process as follows [ 15, 16 ]: ∞ ξt ( ω )= ∑ φτ ,t ( ω ) ζ τ ( ω ), (1)

τ=−∞ where φτ ,t ( ω ), τ , t ∈ Z is a kernel of representation (1), which is real-valued random function of two arguments (or random field on Z2); ζ τ ( ω ), τ ∈ Z is a sequence of independent identically distributed random variables (stationary white noise in the strict sense); random fieldφτ ,t ( ω ) and white noise ζ τ ( ω ) are stochastically independent.

The CLTS representation is valid in the mean-square convergence sense of the series (1).

In the applied problems of information signal analysis or time series forecasting the CLTS (1) is usually considered as a result of sampling or averaging of continuous-time conditional linear random process driven by the process with independent increments, which is infinitely divisible distributed. That is why the white noise in representation (1) also has infinitely divisible distribution and can be specified using one of the known canonical forms. We use the LevyKhintchine form in this article, that is, the stationary white noise ζ τ ( ω ), τ ∈ Z has specified by logarithm of its infinitely divisible characteristic function in the following form: ln f ζ ( u )=iau+ψ ( u ), where function ψ ( u )=−∫∞∞ (eiux−1− 1i+uxx2 ) 1+x2x2 dG ( x ) is uniformly continuous on u∈ R and ψ ( 0 )=0; G ( x ) , x∈ R is a real monotonically non-decreasing and bounded function satisfying the condition G (−∞ )=0; a∈ R, and if mathematical expectation E ζ τ ( ω ) of white ∞ noise is finite then parametera is represented as a= E ζ τ ( ω )−∫ xdG ( x ) . −∞

To represent the expression of m -dimensional characteristic function of CLTS we take into account that it is given on some probability space {Ω , F , P } and defineσ -subalgebra Fφ⊂ F generated by the random function φτ ,t ( ω ) . Also we assume that φτ ,t ( ω ) satisfy the condition ∞ ∑ |φτ ,t ( ω )|< ∞ with probability 1. τ=−∞

Taking into account the above notations and using the results of [ 5, 11 ] the m-dimensional characteristic function f ξ ( u1 , u2 , ... , um ; t1 , t 2 , ... , t m )= E exp[i ∑ uk ξtk ( ω )] of CLTS (1) can m k=1 be represented using the f ξ ( u1 , u2 , ... , um ; t1 , t 2 , ... , t m )= E f ξFφ ( ω , u1 , u2 , ... , um ; t1 , t 2 , ... , t m ) , expression where f ξ φ ( ω , u1 , u2 , ... , um ; t1 , t 2 , ... , t m )= E( exp[i ∑ uk ξtk ( ω )]|Fφ) is conditional with respect m F

k=1 to Fφ characteristic function of CLTS (1), which is expressed as follows: f ξ φ ( ω , u1 , u2 , ... , um ; t1 , t 2 , ... , t m )=exp[ia( ∑ ∑ uk φτ ,tk ( ω ) +¿

∞ m F

τ=−∞ k=1 uk∈ R , t k∈ Z , k =1 , m.

The stationarity condition for CLTS is also similar to the one considered in [ 5, 11 ]. The conditional linear time series is strict sense stationary if the multidimensional distribution of its + ∑ ∫ (exp[ix (∑ uk φτ ,tk ( ω ))]−1−

∞ ∞ m τ=−∞ −∞ k=1

m ix (∑ uk φτ ,tk ( ω )) k=1 1+ x2 kernel doesn’t depend on the same time shift of each argument (that is diagonal shift of the random matrix φτ ,t ( ω ), τ , t ∈ Z).

It means that if random kernels φτ ,t ( ω ), τ , t ∈ Z and φτ +s ,t+s ( ω ) satisfy the condition P (¿ i=1¿ n ¿ j=1 ¿ m {ω : φτ j ,t j ( ω )< xij }¿)= P (¿ i=1¿ n ¿ j=1 ¿ m {ω : φτ j+s ,t j+s ( ω )< xij }¿) , xij∈ R (3) for any s∈ R, then the CLTS (1) is strict sense stationary.

3. Ergodicity and Mixing

In this section we consider the general notion of ergodicity of strict sense stationary time series and its particular cases. Then we justify the conditions for CLTS to be mixing, because mixing implies ergodicity in general sense. It should also be mentioned that mixing property of random process has broader area of application. It can be used for studying the time series complexity and central limit problem [ 17 ], various properties of systems [ 18 ], evaluating the forecasting performance [ 19 ]. The relationships between stationary ergodic and mixing CLRP have been represented on the following Venn diagram (Figure 1).

Let ξt ( ω ) , t ∈ Z be a strictly stationary time series with the values in a measurable space { X , B }. We denote g ( x1 , x2 , ... , xm ), m ≥ 1 a Bm-measurable function and assume that the following expectation exists: E g (ξt1( ω ) , ξt2( ω ) , ... , ξtm( ω ))< ∞ ,∀ t1 , t 2 , ... , t m∈ Z. The time series ξt ( ω ) , t ∈ Z is called ergodic if for any above function g ( x1 , x2 , ... , xm ) the following condition holds with probability 1: lim 1 n n→n∞ ∑t=1 g ( ξt1+t ( ω ) , ξt2+t ( ω ) , ... , ξtm+t ( ω ))= E g ( ξt1( ω ) , ξt2( ω ) , ... , ξtm( ω )) ,∀ t1 , t 2 , ... , t m∈ Z

In the table 1 the particular cases of general ergodicity of stationary time series which are most important for applications in the area of information signal modelling and analysis have been represented. The expressions in last column (ω is omitted for simplicity) holds with probability 1. The corresponding extensions for m ≥ 2 can be obtained like in [ 11 ]. In the table 1 U ( y )={01 ,, yy >≤ 00 is a Heaviside step function [ 20 ].

The mixing property in terms of time series distribution means that the samples (including multivariate) of time series become asymptotically independent when time interval between them tends to infinity. Then joint characteristic function of that samples tends to the product of corresponding characteristic functions [ 11 ].

Following the notations utilized in the paper [ 11 ] we further denote Law ( ξ1( ω ) , ξ2( ω ) , ... , ξm( ω ))= Law ( η1( ω ) , η2( ω ) , ... , ηm( ω )) if two m -dimensional random vectors ( ξ1( ω ) , ξ2( ω ) , ... , ξm( ω )) and ( η1( ω ) , η2( ω ) , ... , ηm( ω )) have the same m -dimensional distribution.

Let random vectors and ( φτ ,t1+t ( ω ) , φτ ,t2+t ( ω ) , ... , φτ ,tm+t ( ω )) ( φτ , s1( ω ) , φτ , s2( ω ) , ... , φτ , sn( ω )) are asymptotically independent if |t|→ ∞, ∀ τ , t1 , t 2 , ... , t m , s1 , s2 , ... , sn∈ Z, that is lim Law (φτ +t ,t1+t ( ω ) , φτ +t ,t2+t ( ω ) , ... , φτ +t ,tm+t ( ω ) , φτ , s1( ω ) , φτ , s2( ω ) , ... , φτ , sn( ω ))=¿ |t|→∞ ¿ Law (φτ ,t1( ω ) , φτ ,t2( ω ) , ... , φτ ,tm( ω )) Law (φτ , s1( ω ) , φτ , s2( ω ) , ... , φτ , sn( ω )) . (5)

Then strict sense stationary CLTS ξt ( ω ) is mixing time series which implies ergodicity in the sense of (4). The proof is analogous to [ 11 ] (but in discrete time) and utilize the above properties of the function ψ ( u ), kernel and characteristic function of conditional linear time series.

4. Conclusions

The conditional linear time series driven by infinitely divisible white noise has been defined. The probability distribution properties of the time series can be analyzed using conditional characteristic functions method. The condition for CLTS to be strict stationary has been represented.

It has been shown that ergodicity and mixing are important characteristics of mathematical model which is used for time series analysis, forecasting, and computer simulation. The continuous-time and discrete-time conditional linear random processes are useful mathematical models in the areas of medical end energy informatics [ 5, 15, 16 ]. That is why the mixing property and ergodicity of strict sense stationary CLRT has been justified using characteristic function method.

The prospective research deals with studying the relationship between the results of this paper and the practically important autoregressive moving average models with random coefficients [ 16 ].

Acknowledgements

This work was partially supported by the NAS of Ukraine within the project "Development of methods and means of monitoring the state of the environment of energy facilities on the basis of wireless sensor networks (Code: Monitoring)".

References

[1] G. E. P. Box , G. M. Jenkins , G. C. Reinsel , G. M. Ljung , Time Series Analysis: Forecasting and Control , 5th ed., Wiley, 2015 .

[2] P. J. Brockwell , R. A. Davis , Introduction to Time Series and Forecasting, 3rd ed., Springer Cham, 2016 .

[3] F. Petropoulos et al., Forecasting: theory and practice , Int. J. Forecast. 38.3 ( 2022 ) 705 - 871 . doi: 10 .1016/J.IJFORECAST. 2021 . 11 .001.

[4] R. J. Hyndman , G. Athanasopoulos, Forecasting: principles and practice , 3rd ed., Melbourne , Australia, OTexts, 2021 .

[5] M. Fryz , B. Mlynko , Properties of Stationarity and Cyclostationarity of Conditional Linear Random Processes , in: Proceedings of the 2020 IEEE 15th International Conference on Advanced Trends in Radioelectronics , Telecommunications and Computer Engineering (TCSET), Lviv-Slavske Ukraine , 2020 , pp. 166 - 170 . doi: 10 .1109/TCSET49122. 2020 . 235415 .

[6] A. Khaleghi , G. Lugosi, Inferring the Mixing Properties of a Stationary Ergodic Process From a Single Sample-Path , IEEE Trans. Inf. Theory 69.6 ( 2023 ) 4014 - 4026 . doi: 10 .1109/TIT. 2023 . 3248375 .

[7] L. Xu , Random Signal Estimation by Ergodicity associated with Linear Canonical Transform , in: 2019 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC) , 2019 , pp. 1080 - 1083 . doi: 10 .1109/APSIPAASC47483. 2019 . 9023088 .

[8] D. G. Kelty-Stephen , M. Mangalam , Fractal and multifractal descriptors restore ergodicity broken by non-Gaussianity in time series , Chaos, Solitons & Fractals 163 ( 2022 ) 112568 . doi: https://doi.org/10.1016/j.chaos. 2022 . 112568 .

[9] A. Baxevani , K. Podgórski , Signals Featuring Harmonics With Random Frequencies - Spectral , Distributional and Ergodic Properties , IEEE Trans. Signal Process . 69 ( 2021 ) 2779 - 2794 . doi: 10 .1109/TSP. 2021 . 3078574 .

[10] Z. Chen , L. Xiao , Q. Wang , Z. Wang , Z. Sun , Coverage control of multi-agent systems for ergodic exploration , in: 2020 39th Chinese Control Conference (CCC) , 2020 , pp. 4947 - 4952 . doi: 10 .23919/CCC50068. 2020 . 9189276 .

[11] M. Fryz , S. Kharchenko , L. Scherbak, Ergodicity and Mixing of Conditional Linear Random Processes in the Problems of Information Signal Modelling and Analysis , in: Proceedings of the 3rd International Workshop on Information Technologies: Theoretical and Applied Problems 2023 (ITTAP 2023 ), Ternopil Ukraine Opole Poland , 2023 , pp. 306 - 314 . URL: https://ceur-ws. org/ Vol- 3628 /short14.pdf.

[12] I. Domowitz , M. A. El-Gamal , A consistent nonparametric test of ergodicity for time series with applications , J. Econom. 102.2 ( 2001 ) 365 - 398 . doi: 10 .1016/S0304- 4076 ( 01 ) 00058 - 6 .

[13] C. Wu , Y. Li , L. Xu , G. Han, Rényi Entropy Rate of Stationary Ergodic Processes , IEEE Trans. Inf. Theory 70.1 ( 2024 ) 1 - 15 . doi: 10 .1109/TIT. 2023 . 3318265 .

[14] H. Wang , C. Wang , Y. Zhao , X. Lin , Toward practical approaches for ergodicity analysis , Theor. Appl. Climatol., 138.3 ( 2019 ) 1435 - 1444 . doi: 10 .1007/s00704-019-02857-1.

[15] V. Babak , A. Zaporozhets , Y. Kuts , M. Myslovych , M. Fryz , L. Scherbak, Models and Characteristics of Identification of Noise Stochastic Signals of Research Objects, in: Proceedings of the 2nd International Workshop on Information Technologies: Theoretical and Applied Problems (ITTAP 2022 ), Ternopil Ukraine, 2022 , pp. 349 - 362 . URL: https://ceur-ws. org/ Vol- 3309 /paper22.pdf

[16] L. M. Scherbak , M. Y. Fryz , V. A. Hotovych , Electricity consumption simulation using random coefficient periodic autoregressive model , in: IOP Conference Series: Earth and Environmental Science , 2023 , 1254 ( 1 ), 012027. doi: 10 .1088/ 1755 -1315/1254/1/012027.

[17] M. Austern , A. Maleki , On the Gaussianity of Kolmogorov Complexity of Mixing Sequences , IEEE Trans. Inf. Theory 66.2 ( 2020 ) 1232 - 1247 . doi: 10 .1109/TIT. 2019 . 2934454 .

[18] T. Adams , A. Quas , Ergodicity and Mixing Properties, in: Encyclopedia of Complexity and Systems Science, R. A. Meyers, Ed., Berlin, Heidelberg, Springer Berlin Heidelberg, 2020 , pp. 1 - 26 . doi: 10 .1007/978-3- 642 -27737-5_ 175 - 3 .

[19] P. Mundt , S. Alfarano , M. Milaković , Exploiting ergodicity in forecasts of corporate profitability , J. Econ. Dyn. Control 111 ( 2020 ) 103820 . doi: https://doi.org/10.1016/j.jedc. 2019 . 103820 .

[20] W. Zhang , Y. Zhou , Chapter 2 - Level-set functions and parametric functions, in: The Feature-Driven Method for Structural Optimization , W. Zhang and Y. Zhou, Eds., Elsevier , 2021 , pp. 9 - 46 . doi: https://doi.org/10.1016/B978-0 -12-821330-8 . 00002 -X.