=Paper=
{{Paper
|id=Vol-3896/short3
|storemode=property
|title=Stationarity, Ergodicity and Mixing Properties of Conditional Linear Time Series Models
|pdfUrl=https://ceur-ws.org/Vol-3896/short3.pdf
|volume=Vol-3896
|authors=Mykhailo Fryz,Svitlana Kovtun,Yurii Kuts,Bogdana Mlynko,Oleksandra Kuchvara
|dblpUrl=https://dblp.org/rec/conf/ittap/FryzKKMK24
}}
==Stationarity, Ergodicity and Mixing Properties of Conditional Linear Time Series Models==
https://ceur-ws.org/Vol-3896/short3.pdf
Stationarity, Ergodicity and Mixing Properties
of Conditional Linear Time Series Models
Mykhailo Fryz 1,2,∗,†, Svitlana Kovtun 2,†, Yurii Kuts 2,3,†, Bogdana Mlynko 1,† and
Oleksandra Kuchvara 4,†
1
Ternopil Ivan Puluj National Technical University, Ruska st. 56, Ternopil, Ukraine
2
General Energy Institute of NAS of Ukraine, Antonovycha st. 172, Kyiv, Ukraine
3
Nat. Techn. University of Ukraine “Igor Sikorsky Kyiv Polytechnic Institute”, Pr. Beresteiskyi 37, Kyiv, Ukraine
4
I. Horbachevsky Ternopil National Medical University, Ternopil, Ukraine
Abstract
We analyze some useful properties of time series which are important in the problems of statistical
data analysis and forecasting. The stationarity of time series can be tested in practice, as there are
many special tests exist, but the property of ergodicity is usually just assumed to be present.
Ergodicity means that the statistical characteristics observed over a single long time series are
representative of the statistical characteristics observed across multiple samples of the process at a
single point in time. The model-based approach has been used in the paper to justify the stationarity
and ergodicity properties of investigated time series. The utilized model is conditional linear time
series with known representation of its characteristic function. It has been used for justifying the
mixing property which implies ergodicity.
Keywords ⋆1
model, signal, conditional linear time series, stationarity, ergodicity, mixing, characteristic function
1. Introduction
Time series analysis is a statistical technique used to analyze data points collected or recorded at
specific time intervals [1 – 3]. The primary goal of time series analysis is to identify patterns,
trends, and other characteristics in the data that can be used for forecasting, monitoring, and
understanding the underlying processes that generate the data [4]. It is widely used in various
fields, including finance, economics, weather forecasting, engineering, medicine, energy, and
environmental science. Very often time series (discrete-time random process) is obtained by
sampling or averaging of continuous-time random process in the problems information signals
and systems modelling, analysis, and estimation.
⋆
ITTAP’2024: 4th International Workshop on Information Technologies: Theoretical and Applied Problems, October 23-
25, 2024, Ternopil, Ukraine, Opole, Poland
1∗
Corresponding author.
†
These authors contributed equally.
mykh.fryz@gmail.com (M. Fryz); kovtunsi@nas.gov.ua (S. Kovtun); y.kuts@ukr.net (Y. Kuts); mlynko@ukr.net
(B. Mlynko); kuchvara@tdmu.edu.ua (O. Kuchvara)
0000-0002-8720-6479 (M. Fryz); 0000-0002-6596-3460 (S. Kovtun); 0000-0002-8493-9474 (Y. Kuts); 0000-0002-
6596-3460 (B. Mlynko); 0000-0002-0248-3224 (O. Kuchvara)
© 2023 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
CEUR
ceur-ws.org
Workshop ISSN 1613-0073
Proceedings
Many real-world time series are non-stationary, requiring transformation before analysis.
But there are enough different methods to test the stationarity (which is invariance of the
probabilistic characteristics of the process over time) property using the real data. Theoretical
analysis of stationarity property of the time series models can be also performed [5].
Ergodicity is another useful property of time series, information signals [6], systems [7–9],
control algorithms [10]. Ergodicity is a concept in statistics and probability theory that describes
the long-term behavior of a system or process. A process is considered ergodic if, over time, its
time-averaged properties converge to its ensemble-averaged properties [11]. This implies that
observing the time series over a long period gives you enough information to understand its
overall behavior, without needing to observe multiple realizations. This is usually practical
since, in many real-world applications, only a single realization of the continuous-time process
or time series is available. The ergodicity is important property also for the problems of time
series forecasting.
But comparing with stationarity, there are only few tests of ergodicity, based on practical
time series analysis. They are related only for some specific classes of random processes, such as
Markov processes [12], [13], or utilize only mean ergodic property [14]. That is why, the ergodic
property of investigated time series usually just assumed.
But there is another approach, which consists in substantiating the mathematical model of
the time series, which is ergodic. For example, continuous-time and discrete-time linear random
processes [15] are ergodic [11].
The stationarity and ergodicity of the important class of continuous-time conditional linear
random processes have been proven in the paper [11]. We develop the ideas [11] of utilizing the
characteristic functions method in present article proving the ergodicity properties for the class
of conditional linear time series (discrete-time conditional linear random processes). The
practical importance of such kind of time series for the information signal mathematical
modelling, estimation, forecasting and computer simulation have been analyzed in [16].
The main goal of the article is to prove the mixing condition for conditional linear time series
because it implies the ergodicity.
Following the general structure and ideas of the paper [11], we analyze further the notion of
stationary conditional linear time series and its multidimensional characteristic functions. Then
we use this tool for proving the mixing and ergodicity consequently.
2. Conditional linear time series and stationarity
We start from conditional linear time series (CLTS) definition and analysis of its characteristic
function which is the tool for stationarity, mixing, and ergodicity proving.
A real-valued conditional linear time series ξ t ( ω ), t ∈ Z , ω ∈ Ω (where Ω is sample space)
is defined as a discrete-time conditional linear random process as follows [15, 16]:
∞
ξ t ( ω )= ∑ φ τ , t ( ω )ζ τ ( ω ), (1)
τ =−∞
where φ τ , t ( ω ), τ , t ∈ Z is a kernel of representation (1), which is real-valued random function
of two arguments (or random field on Z 2); ζ τ ( ω ), τ ∈ Z is a sequence of independent
identically distributed random variables (stationary white noise in the strict sense);
random field φ τ , t ( ω ) and white noise ζ τ ( ω ) are stochastically independent.
The CLTS representation is valid in the mean-square convergence sense of the series (1).
In the applied problems of information signal analysis or time series forecasting the CLTS (1)
is usually considered as a result of sampling or averaging of continuous-time conditional linear
random process driven by the process with independent increments, which is infinitely divisible
distributed. That is why the white noise in representation (1) also has infinitely divisible
distribution and can be specified using one of the known canonical forms. We use the Levy-
Khintchine form in this article, that is, the stationary white noise ζ τ ( ω ), τ ∈ Z has specified by
logarithm of its infinitely divisible characteristic function in the following form:
ln f ζ ( u )=iau+ψ ( u ),
∞
where function ψ ( u )= ∫ e −1−
−∞
(
iux 1+ x 2
iux
1+ x 2 x 2 )
dG ( x ) is uniformly continuous on u ∈ R
and ψ ( 0 )=0; G ( x ), x ∈ R is a real monotonically non-decreasing and bounded function
satisfying the condition G (−∞ )=0; a ∈ R , and if mathematical expectation E ζ τ ( ω ) of white
∞
noise is finite then parameter a is represented as a=E ζ τ ( ω )− ∫ xdG ( x ).
−∞
To represent the expression of m -dimensional characteristic function of CLTS we take into
account that it is given on some probability space { Ω , F , P } and define σ -subalgebra F φ ⊂ F
generated by the random function φ τ , t ( ω ). Also we assume that φ τ , t ( ω ) satisfy the condition
∞
∑ |φ τ , t ( ω )|<∞ with probability 1.
τ =−∞
Taking into account the above notations and using the results of [5, 11] the m-dimensional
[ ]
m
characteristic function f ξ ( u1 , u2 ,... , u m ; t 1 , t 2 , ... , t m )=E exp i ∑ u k ξ t ( ω ) of CLTS (1) can
k
k=1
be represented using the expression
Fφ
f ξ ( u1 , u2 ,... , um ; t 1 , t 2 ,... , t m )=E f ( ω , u1 , u2 ,... , um ; t 1 , t 2 ,... , t m ),
ξ where
( [ ]| )
m
f ( ω , u1 , u2 ,... , um ; t 1 , t 2 , ... , t m )=E exp i ∑ u k ξ t ( ω ) F φ is conditional with respect
Fφ
ξ k
k=1
to F φ characteristic function of CLTS (1), which is expressed as follows:
[ (∑ ∑
∞ m
Fφ
f ( ω , u1 , u2 , ... , um ; t 1 , t 2 ,... , t m )=exp ia
ξ u k φ τ , t ( ω ) +¿
k
τ =−∞ k=1
( [( ( )
) ]
m
ix ∑ u k φ τ , t ( ω )
)]
∞ ∞ m
1+ x 2 (2)
k
+ ∑ ∫ exp ix ∑ u k φ τ , t ( ω ) −1−
k=1
2
dG ( x ) ,
τ =−∞ −∞ k=1
k
1+ x x2
u k ∈ R , t k ∈ Z , k=1 , m.
The stationarity condition for CLTS is also similar to the one considered in [5, 11]. The
conditional linear time series is strict sense stationary if the multidimensional distribution of its
kernel doesn’t depend on the same time shift of each argument (that is diagonal shift of the
random matrix φ τ , t ( ω ), τ , t ∈ Z ).
It means that if random kernels φ τ , t ( ω ), τ , t ∈ Z and φ τ + s , t + s ( ω ) satisfy the condition
P ( ¿ i=1 ¿ n ¿ j=1 ¿ m {ω : φ τ , t ( ω )< x ij }¿ )=P ( ¿ i=1 ¿ n ¿ j=1 ¿ m {ω : φ τ + s , t + s ( ω )< x ij }¿ ) , x ij ∈ R
j j j j
(3)
for any s ∈ R , then the CLTS (1) is strict sense stationary.
3. Ergodicity and Mixing
In this section we consider the general notion of ergodicity of strict sense stationary time series
and its particular cases. Then we justify the conditions for CLTS to be mixing, because mixing
implies ergodicity in general sense. It should also be mentioned that mixing property of random
process has broader area of application. It can be used for studying the time series complexity
and central limit problem [17], various properties of systems [18], evaluating the forecasting
performance [19]. The relationships between stationary ergodic and mixing CLRP have been
represented on the following Venn diagram (Figure 1).
Figure 1: Venn diagram representing the relationships between stationary ergodic and mixing
conditional linear time series
Let ξ t ( ω ), t ∈ Z be a strictly stationary time series with the values in a measurable space
{ X , B }. We denote g( x 1 , x 2 , ... , x m ), m ≥ 1 a B m-measurable function and assume that the
following expectation exists: E g ( ξ t ( ω ), ξ t ( ω ), ... , ξ t ( ω )) <∞ , ∀ t 1 , t 2 , ... , t m ∈ Z . The
1 2 m
time series ξ t ( ω ), t ∈ Z is called ergodic if for any above function g( x 1 , x 2 ,... , x m ) the
following condition holds with probability 1:
lim 1 n
n→∞
n
∑ g( ξ t +t ( ω), ξ t +t ( ω),... , ξ t +t ( ω))=E g( ξ t ( ω), ξ t ( ω),... , ξ t ( ω)), ∀ t 1 , t 2 ,... , t m ∈ Z
1 2 m 1 2 m
t=1
. (4)
In the table 1 the particular cases of general ergodicity of stationary time series which are
most important for applications in the area of information signal modelling and analysis have
been represented. The expressions in last column ( ω is omitted for simplicity) holds with
probability 1. The corresponding extensions for m ≥ 2 can be obtained like in [11].
Table 1
Different types of ergodicity
Ergodicity with
m g( x 1 , x 2 ,... , x m ) t 1 , t 2 , ... , t m Formula (4)
respect to
n
expectation μ 1 g( x )=x t 1=0 lim ∑ ξ t =μ
n → ∞ t=1
covariance g( x 1 , x 2 )=¿ t 1=0, lim 1 n
function R τ
2
¿( x 1−μ )( x 2−μ ) t 2=τ
n→∞
n
∑ ( ξ t −μ )( ξ t +τ −μ )=R τ
t=1
cumulative lim 1 n
distribution 1 g( x )=U ( y−x ) t 1=0 n→∞
n
∑ U ( y−ξ t )=F ξ ( y )
function F ξ ( y ) t=1
characteristic 1 g( x )=e iux
t 1=0 lim 1 n
function f ξ ( u ) n→∞
n
∑ eiu ξ =f ξ ( u )
t
t=1
In the table 1 U ( y )= {01 ,, yy >0≤ 0 is a Heaviside step function [20].
The mixing property in terms of time series distribution means that the samples (including
multivariate) of time series become asymptotically independent when time interval between
them tends to infinity. Then joint characteristic function of that samples tends to the product of
corresponding characteristic functions [11].
Following the notations utilized in the paper [11] we further denote
Law ( ξ 1 ( ω ), ξ 2 ( ω ),... , ξ m ( ω ))=Law ( η1 ( ω ), η2 ( ω ),... , ηm ( ω )) if two m -dimensional
random vectors ( ξ 1 ( ω ), ξ 2 ( ω ),... , ξ m ( ω )) and ( η1 ( ω ), η2 ( ω ),... , ηm ( ω )) have the same
m -dimensional distribution.
Let random vectors ( φ τ , t +t ( ω ), φ τ , t +t ( ω ),... , φ τ , t +t ( ω ))
1 2
and m
( φ τ , s ( ω ), φ τ , s ( ω ),... , φ τ , s ( ω )) are asymptotically independent if |t|→ ∞ , ∀ τ ,
1 2 n
t 1 , t 2 , ... , t m , s1 , s 2 , ... , s n ∈ Z , that is
lim Law ( φ τ +t , t +t ( ω ), φ τ +t , t +t ( ω ),... , φ τ +t , t +t ( ω ), φ τ , s ( ω ), φ τ , s ( ω ),... , φ τ , s ( ω ))=¿
1 2 m 1 2 n
|t|→ ∞
¿ Law ( φ τ , t ( ω ), φ τ , t ( ω ),... , φ τ , t ( ω )) Law ( φ τ , s ( ω ), φ τ , s ( ω ),... , φ τ , s ( ω )) .
1 2 m 1 2 n
(5)
Then strict sense stationary CLTS ξ t ( ω ) is mixing time series which implies ergodicity in
the sense of (4). The proof is analogous to [11] (but in discrete time) and utilize the above
properties of the function ψ ( u ), kernel and characteristic function of conditional linear time
series.
4. Conclusions
The conditional linear time series driven by infinitely divisible white noise has been defined.
The probability distribution properties of the time series can be analyzed using conditional
characteristic functions method. The condition for CLTS to be strict stationary has been
represented.
It has been shown that ergodicity and mixing are important characteristics of mathematical
model which is used for time series analysis, forecasting, and computer simulation. The
continuous-time and discrete-time conditional linear random processes are useful mathematical
models in the areas of medical end energy informatics [5, 15, 16]. That is why the mixing
property and ergodicity of strict sense stationary CLRT has been justified using characteristic
function method.
The prospective research deals with studying the relationship between the results of this
paper and the practically important autoregressive moving average models with random
coefficients [16].
Acknowledgements
This work was partially supported by the NAS of Ukraine within the project "Development of
methods and means of monitoring the state of the environment of energy facilities on the basis
of wireless sensor networks (Code: Monitoring)".
References
[1] G. E. P. Box, G. M. Jenkins, G. C. Reinsel, G. M. Ljung, Time Series Analysis: Forecasting and
Control, 5th ed., Wiley, 2015.
[2] P. J. Brockwell, R. A. Davis, Introduction to Time Series and Forecasting, 3rd ed., Springer
Cham, 2016.
[3] F. Petropoulos et al., Forecasting: theory and practice, Int. J. Forecast. 38.3 (2022) 705–871.
doi: 10.1016/J.IJFORECAST.2021.11.001.
[4] R. J. Hyndman, G. Athanasopoulos, Forecasting: principles and practice, 3rd ed.,
Melbourne, Australia, OTexts, 2021.
[5] M. Fryz, B. Mlynko, Properties of Stationarity and Cyclostationarity of Conditional Linear
Random Processes, in: Proceedings of the 2020 IEEE 15th International Conference on
Advanced Trends in Radioelectronics, Telecommunications and Computer Engineering
(TCSET), Lviv-Slavske Ukraine, 2020, pp. 166–170. doi:10.1109/TCSET49122.2020.235415.
[6] A. Khaleghi, G. Lugosi, Inferring the Mixing Properties of a Stationary Ergodic Process
From a Single Sample-Path, IEEE Trans. Inf. Theory 69.6 (2023) 4014–4026. doi:
10.1109/TIT.2023.3248375.
[7] L. Xu, Random Signal Estimation by Ergodicity associated with Linear Canonical
Transform, in: 2019 Asia-Pacific Signal and Information Processing Association Annual
Summit and Conference (APSIPA ASC), 2019, pp. 1080–1083. doi:
10.1109/APSIPAASC47483.2019.9023088.
[8] D. G. Kelty-Stephen, M. Mangalam, Fractal and multifractal descriptors restore ergodicity
broken by non-Gaussianity in time series, Chaos, Solitons & Fractals 163 (2022) 112568. doi:
https://doi.org/10.1016/j.chaos.2022.112568.
[9] A. Baxevani, K. Podgórski, Signals Featuring Harmonics With Random Frequencies –
Spectral, Distributional and Ergodic Properties, IEEE Trans. Signal Process. 69 (2021) 2779–
2794. doi: 10.1109/TSP.2021.3078574.
[10] Z. Chen, L. Xiao, Q. Wang, Z. Wang, Z. Sun, Coverage control of multi-agent systems for
ergodic exploration, in: 2020 39th Chinese Control Conference (CCC), 2020, pp. 4947–4952.
doi: 10.23919/CCC50068.2020.9189276.
[11] M. Fryz, S. Kharchenko, L. Scherbak, Ergodicity and Mixing of Conditional Linear Random
Processes in the Problems of Information Signal Modelling and Analysis, in: Proceedings of
the 3rd International Workshop on Information Technologies: Theoretical and Applied
Problems 2023 (ITTAP 2023), Ternopil Ukraine Opole Poland, 2023, pp. 306 – 314. URL:
https://ceur-ws.org/Vol-3628/short14.pdf.
[12] I. Domowitz, M. A. El-Gamal, A consistent nonparametric test of ergodicity for time series
with applications, J. Econom. 102.2 (2001) 365–398. doi: 10.1016/S0304-4076(01)00058-6.
[13] C. Wu, Y. Li, L. Xu, G. Han, Rényi Entropy Rate of Stationary Ergodic Processes, IEEE
Trans. Inf. Theory 70.1 (2024) 1–15. doi: 10.1109/TIT.2023.3318265.
[14] H. Wang, C. Wang, Y. Zhao, X. Lin, Toward practical approaches for ergodicity analysis,
Theor. Appl. Climatol., 138.3 (2019) 1435–1444. doi: 10.1007/s00704-019-02857-1.
[15] V. Babak, A. Zaporozhets, Y. Kuts, M. Myslovych, M. Fryz, L. Scherbak, Models and
Characteristics of Identification of Noise Stochastic Signals of Research Objects, in:
Proceedings of the 2nd International Workshop on Information Technologies: Theoretical
and Applied Problems (ITTAP 2022), Ternopil Ukraine, 2022, pp. 349–362. URL:
https://ceur-ws.org/Vol-3309/paper22.pdf
[16] L. M. Scherbak, M. Y. Fryz, V. A. Hotovych, Electricity consumption simulation using
random coefficient periodic autoregressive model, in: IOP Conference Series: Earth and
Environmental Science, 2023, 1254(1), 012027. doi: 10.1088/1755-1315/1254/1/012027.
[17] M. Austern, A. Maleki, On the Gaussianity of Kolmogorov Complexity of Mixing
Sequences, IEEE Trans. Inf. Theory 66.2 (2020) 1232–1247. doi: 10.1109/TIT.2019.2934454.
[18] T. Adams, A. Quas, Ergodicity and Mixing Properties, in: Encyclopedia of Complexity and
Systems Science, R. A. Meyers, Ed., Berlin, Heidelberg, Springer Berlin Heidelberg, 2020,
pp. 1–26. doi: 10.1007/978-3-642-27737-5_175-3.
[19] P. Mundt, S. Alfarano, M. Milaković, Exploiting ergodicity in forecasts of corporate
profitability, J. Econ. Dyn. Control 111 (2020) 103820. doi:
https://doi.org/10.1016/j.jedc.2019.103820.
[20] W. Zhang, Y. Zhou, Chapter 2 - Level-set functions and parametric functions, in: The
Feature-Driven Method for Structural Optimization, W. Zhang and Y. Zhou, Eds., Elsevier,
2021, pp. 9–46. doi: https://doi.org/10.1016/B978-0-12-821330-8.00002-X.