This research focuses on developing an evolutionary neo-fuzzy system for online learning of non-stationary processes. During the study, structural and functional schemes of the system were designed and substantiated. Epanechnikov kernels were proposed as membership functions. Experimental verification of the developed approach demonstrated its effectiveness for forecasting problems under conditions of uncertainty.
The problem of mathematical forecasting of time series data has been well studied. Today, there is a vast number of publications dedicated to this topic, including both theoretical works and practical studies aimed at solving applied problems. Currently, there are many time series forecasting methods, ranging from the simplest, such as regression, correlation, spectral, and exponential smoothing, to more advanced intelligent methods that sometimes rely on rather complex mathematical frameworks [1,2]. The forecasting task becomes significantly more complicated if the analyzed sequences contain trends of an a priori unknown nature, are nonlinear and non-stationary, and include quasi-periodic components, stochastic and chaotic elements, anomalous outliers, and sudden trend jumps. In such situations, nonlinear predictors based on computational intelligence techniques, particularly neurofuzzy systems [3–5], have proven highly effective due to their strong approximation and extrapolation capabilities and ability to adjust parameters based on training data, which is typically given a priori. At the same time, it is assumed that the structure of such a neuro-fuzzy predictor is predefined and does not change during operation and forecasting. The situation becomes significantly more complex when data is received sequentially at a high frequency in the form of a stream, and there is no predefined training set. At the same time the internal structure of the analyzed sequence is a priori unknown and may change over time. Additionally, the internal structure of the analyzed sequence is a priori unknown and may change over time [6,7]. This situation is considered within the theory of evolutionary computational intelligence systems.
Existing evolutionary systems, particularly neuro-fuzzy systems, are still not well adapted for realtime operation under conditions of significant non-stationarity [8–10]. The performance of a forecasting system can be improved by using the so-called neo-fuzzy approach instead of the traditional neuro-fuzzy approach, which has proven effective in time series forecasting tasks [3,11]. However, it was assumed that this sequence changes within a predefined range. At the same time, there is a relatively broad class of real-world processes, primarily in energy, medicine, finance, control, and moving object tracking, where determining the range of the analyzed signal a priori is problematic. This range, in turn, defines the placement of membership functions at the inputs of a neo-fuzzy system.
Therefore, this work proposes an architecture and a fast adaptive learning algorithm for an evolutionary neo-fuzzy system for forecasting significantly non-stationary sequences, where the possible range of variation is a priori unknown, and data is processed sequentially online.
As the basic architecture of the nonlinear predictor, it is convenient to use the so-called ANARX
model (Additive Nonlinear Autoregressive with Exogenous inputs) [12], which has the form (
Here
is the forecast of the analyzed sequence at the current discrete time moment ;
is a specificnonlinear transformation, usually implemented either by an artificial neural network or a neuro-fuzzy system;
is an observed exogenous factor that determines the behavior of the analyzed sequence ; is the model order.
The advantage of model (
Predicator (
In the case of data stream processing, when information arrives in online mode, the primary focus is on the speed of data processing and the simplicity of numerical implementation of the computational intelligence system. Instead of neural networks and neuro-fuzzy systems, which require significant computational resources for their training, it is advisable to use a neo-fuzzy approach [13], which is characterized by high learning speed, computational simplicity, good approximation properties, and the ability to be tuned in an online mode with maximum possible speed. Figure 1 presents the scheme of a neo-fuzzy system designed for real-time prediction of nonstationary processes.
The first layer of the system consists of delay elements (time delay) , which form the historical context of the predicted process the number of delay elements is
. If the predictor structure (
The second hidden layer consists of
neo-fuzzy elements
, each of which is essentially a
neo-fuzzy neuron with two nonlinear synapses
(
The key elements of the system are the nonlinear synapses , which directly solve the
problem of approximating the historical data of the analyzed sequence. Figure 2 shows the structure of
a neo-fuzzy element , which consists of two nonlinear synapses. However, it is important to
note that for the predictor (
It is important to note that each nonlinear synapse essentially performs an F-transform in an adaptive form [14, 15], which makes it a universal approximator of the historical sequence.
Each of the nonlinear synapses contains membership functions , where as well as adjustable synaptic weights must be continuously updated during the processing of the predicted signal.
Upon receiving the input values
equation (
Triangular functions are typically used as membership functions in neo-fuzzy neurons, as they satisfy the conditions of unity partitioning (Ruspini partitioning). [16]. The advantage of triangular functions is that at each training step
, only two neighboring membership functions are activated,
meaning that only synaptic weights require adjustment, which simplifies the learning process. A
drawback of these functions is that they allow only piecewise linear approximation, which reduces
prediction accuracy. In [17], B-splines were used as membership functions, which improved
, which
(
In our opinion, a reasonable compromise between these functions is the use of Epanechnikov kernels, which have proven to be highly effective in regression and pattern recognition tasks [18].
Figure 3 presents the system of membership functions of a nonlinear synapse based on Epanechnikov kernels:
To construct this system, it is necessary to define the range of the controlled sequence and the number of these functions, which is usually chosen based on purely empirical considerations. The number of these functions does not affect the learning process, as only two neighboring functions are activated at any moment. The distance between the extrema of two neighboring functions is determined as shown in equation (5): .
The functions themselves can be expressed as equation (6): , (5) (6) where
The placement of membership functions in nonlinear synapses significantly depends on the a priori defined boundary values , , , , which are usually determined based on purely empirical considerations. When forecasting non-stationary sequences, sudden signal jumps and the emergence of rapidly increasing and decreasing trends that extend beyond the predefined range may occur. Of course, it would be possible to set a sufficiently wide range initially, but this would lead to a significant increase in the number of membership functions and adjustable synaptic weights, making the system excessively complex. We believe that this problem can be addressed by leveraging ideas from evolutionary systems, where not only synaptic weights but also the system's architecture are adjusted during the learning process. Reconfiguring the architecture in real time is quite challenging; therefore, it is advisable to limit modifications to the evolution of the membership function system while preserving their number and the number of synaptic weights. Suppose that at a specific moment in time, the predicted sequence takes a value Figure 4. , as shown in
Based on this value, a new membership function is formed with the center function becomes asymmetric, meaning that (7) holds: , and the
(7) , , (where the indices hL, hR indicate the left and right branches of the bell-shaped membership function ),
If a new predicted signal value
appears, then, similarly, the following membership function is formed, while the function is supplemented by the right branch .
The emergence of new membership functions theoretically should change the structure of nonlinear synapses
and increase the number of adjustable synaptic weights. To prevent this undesirable effect, one can simply replace the function with center in each synapse with the function centered at and continue the process of system forecasting-tuning, where evolution occurs only at the level of nonlinear synapses.
In the case where the analyzed sequence exhibits a decreasing trend, the system’s evolution proceeds similarly.
Suppose that an input signal value
is received. In this case, a membership function is formed with the center
, as shown in Figure 5.
The membership function acquires an asymmetric form such that
, after which a new function is formed
, which replaces the membership function in each nonlinear synapse (8) (9) (10) (11)
Thus, during the forecasting process of significantly non-stationary sequences, the membership function system of the current nonlinear synapse is continuously adjusted. In cases where the exogenous variable is also non-stationary, the system of nonlinear synapses can be similarly adjusted.
Once the membership function system has been formed, it is possible to proceed with adjusting the synaptic weights of the system. Suppose that by the -th moment in time, the prehistory vector of the analyzed sequence and the values of its membership functions are formed as follows (8):
This vector has a dimension of and contains nonzero elements (corresponding to the number of activated membership functions). Next, using equation (9), the vector of synaptic weights is computed, which has the same dimension.
Then, the forecast of the sequence at moment k can be written as follows (10):
After the actual value is received by the system, the synaptic weight vector can be refined using an adaptive learning algorithm [3]: where α is the smoothing parameter.
The newly constructed forecast is then given by
It is easy to see that when , equation (11) takes the form of the Kaczmarz-Widrow-Hoff gradient algorithm, which is optimally fast and best suited for working with non-stationary objects:
For , we arrive at the Goodwin-Remediuk-Kaines stochastic approximation procedure, designed for working with noise-contaminated signals. The trade-off between speed and noise robustness is ensured by varying the parameter
To perform experimental verification and compare the obtained results, we constructed an Evolving Neo-Fuzzy System with triangular membership functions and Epanechnikov functions.
As test data, synthetic time series generated using the following function (13) were used: , , (13) where represents the function values at time , is the primary sinusoidal signal, and is the noise level, determining the intensity of Gaussian noise in the data. Here, is a random variable normally distributed with a mean of 0 and a variance of 1. Three time series variations were considered: a clean sinusoidal signal without noise, a signal with low noise , and a signal with higher noise . The main criterion for evaluating forecast accuracy was the mean relative error (MRE), which allows for assessing the accuracy of predictions for each method.
Initially, forecasting was performed using two kernel membership functions defined over the initial range of time series values ( Figures 6-8. ). The forecasting results are presented in without kernel evolution
As seen in Figures 6-8, both methods demonstrated high accuracy for a pure sinusoidal signal. However, in the presence of noise, the functions proved to be less resistant to fluctuations, leading to an increase in error. The model utilizing Epanechnikov kernels exhibited slightly better smoothing capability at low noise levels, reducing errors compared to triangular functions.
At the next stage, to verify the evolutionary component of the method, the initial interval was reduced ( ), allowing for the simulation of the case where the observed values exceed the predefined range. The forecasting results for this case are presented in Figures 9-11. with kernel evolution
As seen in Figures 9-11, when the initial range of the predicted variable was narrowed, the forecasting accuracy deteriorated compared to the previous numerical experiment. However, the model utilizing Epanechnikov kernels proved to be more effective.
The proposed evolutionary neo-fuzzy system is designed for forecasting significantly non-stationary stochastic and chaotic sequences perturbed by noise in an online mode, where data is processed sequentially in real-time. A key feature of the proposed system is that, during the learning process, not only synaptic weights are adjusted, but also the membership functions, which are represented by Epanechnikov kernels. Moreover, the system can be easily reconfigured in cases where the predicted sequence changes its structure.
The conducted experimental verification has demonstrated the effectiveness of the developed system. Thus, it can be concluded that the proposed approach is characterized by computational simplicity and high processing speed under conditions of non-stationarity and structural uncertainty.
1. Tools and services: GenAI tools were not used in preparation or editing of this work. 2. Tools’ contributions: GenAI tools were not used in preparation or editing of this work.
During the preparation of this work, the authors used Grammarly in order to: Grammar and spelling check. After using this tool, the authors reviewed and edited the content as needed and take full responsibility for the publication’s content.