The electroencephalogram (EEG) modelling and classification methods are very important in medical diagnostics and in creating complex information systems using brain-computer interface-based solutions for Industry 4.0. The mathematical model of the EEG signal has been presented in the paper in the form of a linear random process. The corresponding estimation procedure using the autoregressive model has been considered. The new informative features have been justified as the downsampled kernel of linear random process model representation. The comparative analysis of binary classification machine learning techniques has been performed based on autoregressive coefficients and new extracted informative features. The improvement of classification metrics has been shown.
Electroencephalography (EEG) signals are recordings that capture the electrical potentials of the brain's activity [1, 2]. These measurements are generally non-invasive, using electrodes placed on the scalp, although intracranial electrodes may also be used. Examining the EEG data can yield important information about various brain disorders that affect a wide area of the brain.
Another great area of EEG analysis is related to the creation of complex information systems based on the brain-computer interface (BCI) concept [3, 4], including utilizing the classification of steady-state visual evoked potentials [5]. Several studies, related to EEG-based BCI applications for Industry 4.0 have been presented recently, including e.g. [6, 7]. The authors [6] categorized the possible industrial applications of safety at work, adaptive training, and device control.
Usually, EEG analysis for both medical applications and BCI applications includes mathematical modelling, features extraction, and classification methods. Linear models and techniques are significant in the field of EEG signal processing [1]. Discrete-time univariate and multivariate linear random sequences in the form of autoregressive moving average models are especially important and widely used [8, 9, 10]. Moreover, the continuous-time linear random process [11, 12] is the biophysically reasonable mathematical model of EEG representing the signal as the sum of many stochastically independent excitatory and inhibitory postsynaptic potentials which are generated by the pyramidal neurons in the cerebral cortex in response to the spike train [2] that can be considered as the Poisson arrival process. Another useful EEG signal model is a conditional linear random process considering the stochastic dependence between the postsynaptic potentials forming the EEG [2, 12]. All the above linear models can be analyzed using the characteristic function methods.
Staying within the linear paradigm it is important to extract the new diagnostic features and find modern machine-learning classification techniques to improve the classification performance. Because of main elements of a linear random process is the kernel of its representation and generative white noise, the idea is to consider the first one as the source of informative characteristics for EEG analysis. We consider only binary classification problems in our paper.
The primary objective of the paper is to justify the informative features of EEG signals based on the kernel of its representation as a linear random process and to investigate the machine learning classification techniques based on the extracted features. 2. Methodology of modelling, features extraction and estimation
Continuous-time linear random process ξ(ω,t), t ∈ (−∞, ∞), given on some probability space {Ω, F, P} is defined using the following integral representation [12]:
∞
ξ(ω,t) = ∫ ϕ(τ,t)dη(ω, τ), ω∈ Ω, t ∈ (−∞, ∞) , (
−∞
where η(ω, τ), τ ∈ (−∞, ∞) , P(η(ω, 0) = 0) = 1 is a stochastically continuous random process with
independent increments;
ϕ(τ,t), τ,t ∈ (−∞, ∞) is a nonrandom function (kernel of representation (
The linear random process (
The discrete-time counterpart of linear random process (LRP) (
∞
ξt (ω) = ∑ ϕτ,tζτ (ω) , (
τ=−∞
where ζτ (ω), τ ∈ Z is a sequence of independent infinitely divisible random variables (infinitely
divisible strict-sense white noise) with finite expectation and variance Eζτ (ω) =aτ , Var [ζτ (ω)] =στ2 ;
ϕτ,t , τ,t ∈ Z is a nonrandom function (kernel) satisfying the following conditions:
∞ ∞
∑ ϕτ,t aτ < ∞ , ∑ ϕτ,t 2 στ2 < ∞ , ∀t ∈ Z
τ=−∞ τ=−∞
(the conditions are important to guarantee the mean-square convergence of the stochastic series (
The expectation Eξt (ω) and covariance function Rt1,t2 , t1,t2 ∈ Z of linear random process (
τ=−∞ ∞ Rt1,t2 = ∑ ϕτ,t1ϕτ,t2 στ .
2 τ=−∞
If ζτ (ω) , τ ∈ Z is strict-sense stationary white noise, and the kernel ϕτ,t depends only from the
difference of its arguments, that is ϕτ,t =ϕt−τ , then LRP (
∞ ∞ ∞ ξt (ω) = ∑ ϕt−τζτ (ω) = ∑ ϕτζt−τ (ω), ∑ ϕτ2 < ∞ .
τ=−∞ τ=−∞ τ=−∞ The expectation and covariance function of the stationary LRP are represented as:
∞ ∞ Eξt (ω) = a ∑ ϕτ , Rs = σ2 ∑ ϕτϕτ+s , s ∈ Z ,
τ= −∞ τ= −∞ where Eζτ (ω) =a , Var [ζτ (ω)] =σ2 .
Among linear random processes the autoregressive models are the most important for applied problems of signal processing, especially when it is necessary to identify, predict, simulate, or classify the stochastic signals [13, 14].
In most important for practical application cases, a stationary discrete-time linear random process can be represented as a linear stochastic difference equation which is known as autoregressive moving average model of the order ( p, q) : where ak , k =p; 1, bk , k =qare 0, the real coefficients.
If q = 0 and b0 = 1 we obtain the autoregressive (AR) model of the order p , which is represented in the following form:
p ξt (ω) = −∑ ak ξt−k (ω) + ζt (ω), t ∈ Z ,
k =1
p q
ξt (ω) =−∑ ak ξt−k (ω) + ∑ bk ζt−k (ω), t ∈ Z ,
k =1 k =0
(
It is well known [13, 14] that covariance function of the stationary autoregressive model satisfies the following recurrent relation: p −∑ ak R−k + σ2 , τ =0, k =1 Rτ = −∑p ak Rτ−k , τ > 0.
k =1 R0 R1 R 2 Rp
R0 R1 Rp−1
R−1 R0 Rp−2
R− p 1 σ2
R− p+1 a1 0 R− p+2 a2 = 0 .
R0 ap 0
Representing the above equation for τ =0, p the system of linear equations is obtained, the matrix expression of the system is called Yule-Walker equations for AR model:
R−1 R−2
The Yule-Walker equations system (
Our idea is to use the samples of the kernel of stationary discrete-time LRP representation of EEG, because, as it follows from the biophysical reasoning, the kernel represents the integral properties of the postsynaptic potentials forming EEG signal.
The general methodology of EEG analysis then consists of the following steps:
• EEG signal acquisition and preprocessing,
• order estimation of AR model [15],
• covariance function Rτ , τ =0, p estimation,
• solving the linear equations (
The extracted and estimated parameters can be used then in the algorithms of statistical hypotheses testing or machine learning classification [16]. The last approach is considered in the next section of the paper. We evaluate and compare the performance of binary classification algorithms for the case of using AR parameters as informative features versus set of samples of the kernel of LRP representation while keeping the same dimensionality of the features space.
Several binary classification techniques have been compared using the dataset which is described in [17]. The length of each EEG realization is 23.6 sec, and it consists of 4097 samples. The realizations have passed the wide sense stationary test, and the eye movement and muscle activity artefacts have been removed. The binary classification tests have been analyzed based on healthy persons with open eyes (Class A) and closed eyes (Class B). The dataset consists of 100 elements in each of the classes. The representatives of Class A and Class B have been shown in Figure 1.
The covariance function estimations of open eyes EEG and closed eyes EEG have been represented in Figure 2. The above realizations have been utilized to calculate the estimations.
The kernel of EEG representation in the form of discrete-time linear random process has been estimated as the impulse response of corresponding recursive filter related to autoregressive model of the order 4. The coefficients of autoregressive model have been estimated using Yule-Walker approach. The estimations (based on realizations which have been shown in the Figure 1) of the kernel for open eyes EEG and closed eyes EEG have been represented in Figure 3.
The performance measures of different binary classification algorithms [18, 19, 20] based on AR parameters as informative features have been calculated using a 10-fold cross-validation approach and represented in Table 1. The best results are highlighted.
Comparing the above results, we can conclude that Extra Trees Classifier based on the second features set outperforms the other considered algorithms based on the first features set.
The analysis of the above results shows that the kernel of linear random process representation of EEG signal can be used as an informative feature in the problems of its binary classification using machine learning techniques. We used the number of kernel samples equal to the order of the
The performance measures of the same classifiers based on the set of kernel samples {ϕ1, ϕ5 , ϕ9 , ϕ13} as informative features have been calculated using a 10-fold cross-validation approach and represented in Table 2. The best results are highlighted. corresponding autoregressive model to compare the classification performance of different feature sets of the same dimensionality. The study of the dependence of classification accuracy on the number of informative features is a goal of future research. Still, we suppose that the dimensionality of the feature space should be equal to the order of the autoregressive model, but the kernel samples included in this space should be justified.
The main result of our research is the justification of new information features and corresponding binary classifier (the extra trees classifier) that can be used in medical diagnostics and in the problems of brain-computer interface based on EEG analysis. The results can be further extended to the alternative case of the autoregressive process with random coefficients [2], taking into account the nonlinear dynamics of EEG.
Being an ensemble method, the extra trees classifier combines predictions from multiple decision trees to make the final classification. This ensemble approach improves generalization and reduces the risk of overfitting, leading to better performance and more reliable results. There are also other advantages, such as fast training speed, efficient parallelization, and robustness to noisy data. It is important in the context of this paper that the extra trees classifier provides a measure of feature importance, gaining insights into the relevance and contribution of each feature to the classification task. This information can be useful for future research for feature selection, identifying the most influential kernel samples.
Methodology of features extraction and their estimation for EEG analysis based on the mathematical model in the form of linear random process has been considered. The set of informative characteristics has been identified consisting of the samples of the kernel of linear random process.
The new set of informative features has been compared with the traditional classification characteristics which are the autoregressive coefficients. Several binary classification machine learning algorithms have been analyzed for the task of classification of open eyes EEG and closed eyes EEG. Extra Trees Classifier has appeared to perform the best with the set of new informative features.
The prospective research should be related to the formal justification of the task of certain samples selection of the kernel which should be used as the classification features.
Model and Wavelet Packet Decomposition, Neural Process. Lett. 45.2 (2017) 365–378. doi: 10.1007/s11063-016-9530-1. [9] Z. Liu, L. Wang, S. Xu, K. Lu, A multiwavelet-based sparse time-varying autoregressive modeling for motor imagery EEG classification, Comput. Biol. Med. 155 (2023) 106196. doi: 10.1016/j.compbiomed.2022.106196. [10] C.-S. Ouyang, R.-C. Yang, C.-T. Chiang, R.-C. Wu, L.-C. Lin, EEG autoregressive modeling analysis: A diagnostic tool for patients with epilepsy without epileptiform discharges, Clin.
Neurophysiol. 131.8 (2020) 1902–1908. doi:10.1016/j.clinph.2020.04.172. [11] M. Fryz, Mixing property and ergodicity of linear random processes, Proceedings of the 2009 IEEE International Workshop on Intelligent Data Acquisition and Advanced Computing Systems: Technology and Applications, IEEE, Rende Italy, 2009, pp. 343-346. doi: 10.1109/IDAACS.2009.5342967. [12] V. Babak, A. Zaporozhets, Y. Kuts, M. Myslovych, M. Fryz, L. Scherbak, Models and Characteristics of Identification of Noise Stochastic Signals of Research Objects, in: Proceedings of the 2nd International Workshop on Information Technologies: Theoretical and Applied Problems (ITTAP 2022), Ternopil Ukraine, 2022, pp. 349-362. URL: https://ceur-ws.org/Vol3309/paper22.pdf [13] G. E. P. Box, G. M. Jenkins, G. C. Reinsel, G. M. Ljung, Time Series Analysis: Forecasting and
Control, 5th ed., Wiley, 2015. [14] P. J. Brockwell, R. A. Davis, Introduction to Time Series and Forecasting, 3rd ed., Springer
Cham, 2016. [15] S. Ross, Introduction to Probability Models, 12th ed., Academic Press, 2019. [16] J. J. Li, X. Tong, Statistical Hypothesis Testing versus Machine Learning Binary Classification:
Distinctions and Guidelines, Patterns 1.7 (2020) 100115. doi:10.1016/j.patter.2020.100115. [17] R. G. Andrzejak, K. Lehnertz, F. Mormann, C. Rieke, P. David, C. E. Elger, Indications of nonlinear deterministic and finite-dimensional structures in time series of brain electrical activity: Dependence on recording region and brain state, Phys. Rev. E 64.6 (2021) 061907. doi: 10.1103/PhysRevE.64.061907. [18] A. Tharwat, Classification assessment methods, Appl. Comput. Informatics 17.1 (2021) 168– 192. doi:10.1016/j.aci.2018.08.003. [19] S.-C. Huang, T.-H. Le, Chapter 3 - Binary classification problem, in: Principles and Labs for Deep Learning, S.-C. Huang, T.-H. Le, Eds., Academic Press, 2021. doi:10.1016/B978-0-32390198-7.00003-3. [20] T. Marwala, W. T. Mongwe, R. Mbuvha, 2 - Sampling benchmarks and performance metrics, in: Hamiltonian Monte Carlo Methods in Machine Learning, T. Marwala, W. T. Mongwe, R. Mbuvha, Eds., Academic Press, 2023. doi:10.1016/B978-0-44-319035-3.00014-8.