In this paper new class of deep learning - cascade neo-fuzzy neural network (CNFNN) was considered and investigated. Neo-fuzzy neuron with two inputs is used as a node of hybrid network. The experimental investigations were carried out during parameters of neo-fuzzy network were found: number of inputs, training/test sample ratio and number of linguistic variables. Method GMDH was used for optimal structure construction of deep hybrid network. The experimental investigations of deep NFNN were carried out in the problem of market index forecasting at German stock exchange and Google share prices. The comparison experiments of the deep neo-fuzzy network with alternative methods GMDH and conventional cascade neo-fuzzy network were carried out and the efficiency of suggested hybrid NFN was estimated Deep learning, GMDH, cascade neo-fuzzy network, parameters and structure optimization.
Nowadays forecasting problems of share prices and market indicators attract great attention of investors and managers of invest funds. For forecasting at financial markets usually are applied methods of regressive analysis ARIMA, ARCH and GARCH methods, exponential smoothing [1].
But last years for solution of this problem fuzzy neural networks (FNN) were suggested [5-7]. Their main advantages are capability to work with fuzzy, incomplete and qualitive information and to utilize expert knowledge. Besides, FNNs have properties of high approximation due to FAT theorem [5] and interpretability. But for application of FNN in forecasting problems its necessary to train rule base and membership functions of fuzzy rules. This demands large computational resources and a lot of training time.
Last years new class of FNN- cascade neo-fuzzy neural networks (CNFNN) appeared [8], their main advantage is absence of necessity to train membership functions and only rule weights are to be trained using input sample. This enables to substantially cut training time and computational expense and to apply this class of FNN for high dimensional problems (Big Data).
Usually training of NN
means the adjustment of weights between neurons. But efficiency of training can be substantially improved if to adapt not only neuron weights, but a network structure as well using training sample. For this aim the application of Group Method of Data Handling seems very promising. GMDH is based on the principle of self- organization and enables to construct structure of model in the process of its run. Besides, as building blocks for model simple sub-models called partial descriptions consisting of two variables are used [2-4]. That allows to GMDH to work with short training samples.
2021 Copyright for this paper by its authors.
For the first time application of GMDH to construct structure of neural networks and to train its weights was suggested by A.G. Ivakhnenko and his collaborates (so-called “active neurons”) [2-4]. In the next works method GMDH was successfully applied for construction of hybrid neuro-fuzzy networks with kernel activation functions [9], spiking neurons [10], wavelet functions [11,12] and other class of fuzzy neural networks [13]. But in these works, the important property of GMDH – application of basic models with only two inputs and small number of tunable parameters wasn’t used. This property is very important for deep learning fuzzy networks and enable to cut number of adapted parameters and training time as well.
The goal of this paper is to find optimal parameters of deep CNFNN, construct its structure using GMDH, investigate its efficiency in the forecasting problem of share prices at stock exchange and to compare its efficiency with CNFNN of standard structure.
The goal of experiments was to analyze the forecasting efficiency of deep learning neo-fuzzy neural network (NFNN) in the problem of forecasting share prices and market index of German stock exchange DAX, in particular find optimal parameters of neo-fuzzy network – number of inputs, number of fuzzy membership functions and optimal structure of deep NFNN.
As input data were taken average month values of market index DAX in the period since January 2010 to December 2016. Then total sample size was 80 elements – average month values. The input data is presented in the Table 1.
Network training was performed by gradient descent method with adaptive steps and WidrowHoff method (6) in the sequential mode (see previous section). The goal of experiments was to find optimal parameters NFNN. In experiments the following parameters varied: inputs number (prehistory length), number of layers, number of linguistic variables (fuzzy sets/per variable), rules number and training/ test sample ratio Ntrain /N test (%).
In the first experiment the number of layers was varied under different ratio training test sample and its influence on forecasting accuracy was explored. The corresponding results are presented in the Table 2. In denoting CNFNN (m,n,k) the first digit m indicates number of layers, the second digit n-inputs number, the third k-number of linguistic variables.
As it follows from the presented results the optimal inputs number exists which, in general case depends on ratio N train/N test . The optimal inputs number for ratio N train/N test = 50 equals to 4. 30 25 20 15 10 5 0 25 20 15 10 5 0
CNFNN(2,4,4)
CNFNN(3,4,4)
CNFNN(4,4,4)
CNFNN(5,4,4) As it follows from Figure 1 the optimal layers number for considered problem is equal to 4.
Further investigation of MAPE dependence on inputs number was carried out. The corresponding results are presented in Figure 2 for ratio Ntrain/N test =50/50.
The important parameter for deep NFN is a number of linguistic variables (fuzzy sets per one variable). The corresponding investigations were carried out and the results are presented in the Figure 3.
The presented results show optimal number of linguistic variables is equal to 4 for the considered problem.
The investigations of ratio N train/N test dependence on forecasting accuracy were carried out.
The corresponding results of forecasting accuracy versus number of layers for different rations Ntrain/N test are presented in figure 6 and in the Table 3.
The found dependence of criterion MAPE on inputs number for different ratios N train/N test are presented in the Table 4.
The forecasting accuracy versus number of linguistic variables was also investigated for different ratios Ntrain /Ntest and presented in Figure 5.
CNFNN(4,5,2) CNFNN(4,5,3) CNFNN(4,5,4) CNFNN(4,5,6)
As it follows from the presented results for each class of financial processes there exists optimal number of layers of NFNN. Under its further increase criterion MAPE on test sample begins to increase or stops to change. That is well comply with principle of self-organization of GMDH [2-4]. The similar dependence was detected for number of inputs and number of linguistic variables.
Further the similar forecasting experiments were carried out with FNN ANFIS. Number of inputs and number of linguistic values were taken 4. The efficiency comparison with the deep neo-fuzzy network with similar parameters values NFNN(4,4,4) was performed. The corresponding results for both networks are presented in the Table 5.
Deep NFNN(4,4,4) 50-50 19,7% 15,5% 60-40 17,65% 15,4% 70-30 12,54% 9,6584%
80-20 6,9554% 4,4325%
90-10 4,5614% 3,1952%
As the results of comparison show the deep neo-fuzzy neural network has higher forecasting accuracy than ANFIS. Additional advantages of NFNN are less computational complexity and less training time due to lack of necessity to adjust membership functions These properties enable to use NFNN in Big Data forecasting problems.
In the process of investigations GMDH was applied to construction of optimal structure of hybrid cascade network. In this research Google shares close prices since August till December 2019 were forecasted. The process of hybrid network structure generation which was obtained by GMDH algorithm is presented in Figure 6 [14].
The optimal structure generated by GMDH was such: 6 neurons (A0, A1, A2, A3, A4, A5) at the first layer, 4 neurons (B0, B1, B2, B3) at the second layer, 2 neurons (C0,C1) at the third layer and one neuron (D0) at the last layer. All 5 inputs were used in the structure.
1. In this paper new class of deep learning networks-hybrid cascade neo-fuzzy neural network (NFNN) based on GMDH was developed and explored in the problem of forecasting market indicator of German stock exchange and Google share prices. In this type of deep networks neofuzzy neuron with two inputs is used as a node.
The experimental explorations were carried out during which the optimal parameters of hybrid neo-fuzzy network were determined. 2. The problem of optimal structure generation of hybrid cascade network was considered and for its solution method GMDH was applied and investigated. 3. The comparative experiments of the deep hybrid network with alternative methods GMDH and cascade network were carried out and forecasting efficiency of the suggested hybrid network was estimated and proved to be the best one. 4. After experiments it was detected the developed deep hybrid neo-fuzzy network is very promising for forecasting in the financial sphere. Besides, it’s free from typical drawbacks of conventional deep learning networks.
[1] L. Lewis, Methods of forecasting economic indicators (transl. from English). Finance and
Statistics, Moscow, 1986. [2] A.G. Ivakhnenko, G.A. Ivakhnenko, J.A. Mueller, Self-organization of the neural networks with active neurons. Pattern Recognition and Image Analysis 4, 2 (1994): 177-188. [3] A.G. Ivakhnenko, D. Wuensch, G.A. Ivakhnenko, Inductive sorting-out GMDH algorithms with polynomial complexity for active neurons of neural networks. Neural Networks 2 (1999): 1169-1173. [4] G.A. Ivakhnenko, Self-organization of neuronet with active neurons for effects of nuclear test explosions forecasting. System Analysis Modeling Simulation 20 (1995): 107-116. [5] M. Zgurovsky, Yu. Zaychenko, Fundamentals of computational intelligence- System approach.
Springer, 2016. [6] L.-X. Wang, J. M. Mendel, Fuzzy basis functions, universal approximation, and orthogonal leastsquares learning. IEEE Trans. on Neural Networks 3, №5 (1992): 807-814. [7] J.-S. Jang, ANFIS: Adaptive-network-based fuzzy inference systems. IEEE Trans. on Systems,
Man, and Cybernetics. 23 (1993,): 665-685. [8] T. Yamakawa, E. Uchino, T. Miki, H. Kusanagi, A neo-fuzzy neuron and its applications to system identification and prediction of the system behavior, in: Proceedings 2nd Intеrn. Conf.
Fuzzy Logic and Neural Networks «LIZUKA-92». Lizuka, 1992, pp. 477-483. [9] Ye. Bodyanskiy, N. Teslenko, P. Grimm, Hybrid evolving neural network using kernel activation functions, in: Proc. 17th Zittau East-West Fuzzy Colloquium, Zittau/Goerlitz, HS, 2010, pp. 39-46. [10] Ye. Bodyanskiy, O.A .Vynokurova, A.I. Dolotov, Self-learning cascade spiking neural network for fuzzy clustering based on Group Method of Data Handling. J. of Automation and Information Sciences, 45, №3 (2013,): 23-33. [11] Ye. Bodyanskiy, O. Vynokurova, A. Dolotov, O. Kharchenko, Wavelet-neuro-fuzzy network structure optimization using GMDH for the solving forecasting tasks, in: Proceedings 4th Int.
Conf. on Inductive Modelling ICIM 2013, Kyiv, 2013, pp. 61-67. [12] Ye. Bodyanskiy, O. Vynokurov, N. Teslenko, Cascade GMDH-wavelet-neuro-fuzzy network, in: Proceedings 4th Int. Workshop on Inductive Modeling «IWIM 2011», Kyiv, Ukraine, 2011, pp. 22-30. [13] Ye. Bodyanskiy, O. Boiko, Yu. Zaychenko, G. Hamidov, Evolving Hybrid GMDH-Neuro-Fuzzy Network and Its Applications, in: Proceedings of the Intern conference SAIC 2018, Kiev, Ukraine, 2018. [14] E. Bodyanskiy, Yu. Zaychenko, O. Boiko, G. Hamidov, A. Zelikman, The hybrid GMDH-neofuzzy neural network in forecasting problems in financial sphere, in: Proceedings of the International conference IEEE SAIC 2020, Kiev, Ukraine, 2020.