Methods in the Short- Term and Middle-Term Forecasting in Financial Sphere

Abstract

In this paper the problems of short- and middle-term forecasting at the financial sphere are considered. For this problem intelligent forecasting methods: LSTM and hybrid deep learning networks based on GMDH are suggested. The optimal parameters of LSTM and hybrid networks were found. Optimal structures of hybrid networks were constructed for short-term and middle-term forecasting. The experimental investigations of LSTM and hybrid DL networks were carried out and their accuracy was compared. The fields of preferable application of LSTM and hybrid DL networks in forecasting problems in finance are determined. Short-term, middle-term financial forecasting, LSTM, hybrid DL network, optimization Problems of forecasting share prices and market indexes at stock exchanges pay great attention of investors and various money funds. For their solution were developed and investigated powerful intelligent methods and technologies among them neural networks and fuzzy logic systems.

1. Introduction. Analysis of previous works

hybrid DL networks based on GMDH

method [ 11 ]. The application self-organization in these networks enables to train not only neuron weights but to construct optimal structure of a network. Due to method of training in these networks weights are adjusted not simultaneously but layer after layer. That prevents the phenomenon of vanishing or explosion of gradient. It’s very important for networks with many layers.

The first works in this field used as nodes of hybrid network Wang-Mendel neurons with two inputs [ 11 ]. But drawback of such neurons is the demand to train not only neural weights but the parameters of fuzzy sets in antecendents of rules as well. That needs lot of calculational expenses and large training time as well. Therefore, later DL neo-fuzzy networks were developed in which as nodes are used neo-fuzzy neurons by Yamakawa [ 12,13 ]. The main property of such neurons is that it’s necessary to train only neuron weights not fuzzy sets. That demands less computations as compared with Wang-Mendel neurons and significantly cuts training time as a whole. The investigation of both classes of hybrid DL networks were performed and their efficiency at forecasting in financial sphere was compared in [ 13 ]. Therefore, it presents a great interest to compare the efficiency of hybrid DL networks and LSTM at the problem of forecasting at financial sphere. The goal of this paper is to investigate the accuracy of hybrid DL networks and LSTM at the problem of forecasting market indices at the stock exchange, compare their efficiency at the different intervals and to determine the

2022 Copyright for this paper by its authors. classes of forecasting problems for which the application of corresponding computational intelligence technologies is the most perspective.

2. The description of the evolving hybrid GMDH-neo-fuzzy network

The evolving hybrid DL-network architecture is presented in Fig.1. To the system’s input layer a ( × 1)-dimensional vector of input signals is fed. After that this signal is transferred to the first hidden layer. This layer contains nodes, and each of these neurons has only two inputs.

At the outputs of the first hidden layer the output signals are formed. Then these signals are fed to the selection block of the first hidden layer. precise signals by some chosen criterion (mostly by the mean squared error 2

It selects among the output signals ̂ [ 1 ] ∙ 1 ∗ ( 1 ∗= is so called freedom of choice) most best outputs of the first hidden layer ̂[ 1 ] ∗∙ 2 pairwise combinations ̂[ 1 ] ∗, ̂ [ 1 ] ∗ are formed. These signals are fed to the second hidden layer, that is formed by neurons [ 2 ]. After training these neurons output signals of this layer ̂[ 2 ] are transferred to the selection block [ 2 ] which choses best neurons by accuracy (e.g. by the value of 2

[ 2 ]) if the best signal of the second layer is better than the best signal of the first hidden layer ̂1[ 1 ] ∗. Other hidden layers forms signals similarly. The system evolution process continues until the best signal of the selection block [ +1] appears to be worse than the best signal of the previous s th layer. Then we return to the previous layer and choose its best node neuron [ ] with output signal ̂[ ]. And moving from this neuron (node) along its connections backwards and sequentially passing all previous layers we finally get the structure of the GMDH-neo [ 1 ] ). Among these 1 ∗ fuzzy network.

It should be noted that in such a way not only the optimal structure of the network may be constructed but also well-trained network due to the GMDH algorithm. Besides, since the training is performed sequentially layer by layer the problems of high dimensionality as well as vanishing or exploding gradient are avoided. 2.1.

Neo-fuzzy neuron as a node of hybrid GMDH-system

Let’s introduce into consideration the architecture of the node that is presented in Fig.2 and is suggested as a neuron of the proposed GMDH-system. As a node of this structure a neo-fuzzy neuron (NFN) by Takeshi Yamakawa and co-authors in [ 9 ] is used. The neo-fuzzy neuron is a nonlinear multi-input single-output system shown in Fig.2. The main difference of this node from the general neo-fuzzy neuron structure is that each node uses only two inputs. It realizes the following mapping: 2 nonlinear synapses which perform transformation of input signal in the form where is the input ( = 1,2,…, ), ̂ is a system output. Structural blocks of neo-fuzzy neuron are and realize fuzzy inference: if is

then the output is ,where is a fuzzy set which membership function is , is a synaptic weight in consequent [ 11 ].

( )= ∑ ( ) (2) (3) (4) (5)

2.2. The neo-fuzzy neuron learning algorithm

The learning criterion (goal function) is the standard local quadratic error function: ( )= 1 2

1 1 ( ( )− ̂( ))2 = 2 ( )2 = ( ( )− ∑ 2

∑ ( ( ))) 2 ℎ conventional least squares method [ 12 ] =1 + =1 =1 (real value). used in the form [ 11,12 ] where (•)+ means pseudo inverse of Moore-Penrose (here ( )denotes external reference signal If training observations are fed sequentially in on-line mode, the recurrent form of the LSM can be [ 1 ]( )= (∑ [ 1 ]( ) [ 1 ] ( )) ∑ [ 1 ]( ) ( )= [ 1 ]( )∑ [ 1 ]( ) ( ) ( )= ( − 1)+

( )= ( − 1)− { ( − 1)( ( )− ( ( − 1)) ( ( ))) ( ( )) 1+ ( ( ( ))) ( − 1) ( ( )) ( − 1) ( ( ))( ( ( ))) ( − 1)

1+ ( ( ( ))) ( − 1) ( ( )) .

3. LSTM model and architecture

Recurrent neural networks (RNNs) are based on the idea of passing through sequence of time steps with derivatives that do not explode or vanish. The idea is that some gated self-loop can be introduced that allows to decide what information should be forgotten, saved, and kept. That decision should be based on features that neural networks learn during training. One of the most successful architectures that implement that idea is Long-Short-Term-Memory (LSTM) recurrent neural network [ 1-5 ].

LSTM replaces the regular RNN unit with LSTM block that has its internal memory and can be

blocks. LSTM has two types of connections – external connections that are similar to the recurrent connection between RNN hidden units, and internal state ( ) that has a recurrent internal connection to itself. In addition, LSTM block has three main gates that control an information flow and decide whether new information should be forgotten or saved and whether we need to keep old information in memory [ 5 ]. An example of LSTM architecture is shown in Fig.3. that contains information from LSTM block outputs in the previous time steps; is forget gate bias vector; is a matrix of input weights for forget gate; is a matrix of recurrent weights for forget gate.

The next step for LSTM block consists of several intermediate steps. First, the input gate decides which information in the internal state should be updated with new data. Then, the network creates a list of new elements that reflect new information that should be added to the internal state. Finally, the network combines all information from previous steps and updates the internal state ( ). All these operations are described with the following equation [ 4, 5 ]: ( ) = ( ) ( −1)+ ( ) ( + ∑ , ( )+ ∑ , ℎ

The last step of LSTM block decides which information should be returned as output. Output value calculated using output gate mechanism: ℎ( ) = ℎ( ( ) ( )) ( ) = ( + ∑ , ( )+ ∑ , ℎ where , , respectively bias vector, input, and recurrent weights matrices of output gate.

For training LSTM stochastic gradient method and its modern modifications are used. LSTM architecture has been successful on real-world tasks in different domains and shows that it works much better with long-term dependencies than poor RNNs [ 4, 5 ].

4. Experimental investigation and results analysis 4.1.

Data set

As an input data was taken corporate Emerging Markets Bond total Return Index (EMBRI) at NASDAQ stock exchange at the period since January till August 2022. The sample consisted of instances which were divided on training and test subsamples.

Flow chart of EMBRI is presented in the Fig. 4.

Experimental investigations of hybrid DL networks

The first series of experiments were performed with hybrid Deep Learning network with neo-fuzzy neurons as nodes. At experiments the following parameters were varied: ratio training/test sample, number of inputs 3-5, number of fuzzy sets per variable 3-5 and membership functions: Bell, Gaussian and Triangular. The goal of experiments was to find optimal parameters values. Forecast period was taken 5 days, as the accuracy metrics were taken MSE and MAPE. In the first experiment Bell MF was explored. After experiment optimal parameters were found for the hybrid DL network: number of inputs – 3, number of fuzzy sets – 3, ratio – 0.8. With these parameters values the best accuracy at the test sample was attained: MSE = 0,424, MAPE = 0,155.

The next experiments were performed for hybrid network with Gaussian MF. After experiments were found optimal parameters of hybrid DL network: number of inputs – 3, number of rules – 4, ratio training/test – 0.6. With these parameters the forecasting results are presented in the Table 1.

In the Table 2 the process of structure generation of the best network is presented with optimal parameters. As it follows from Table 2 optimal structure consists of 3 layers: 3 nodes at the first layer, 3 nodes at the second layer and 1 node at the third layer. Flow chart of forecasting with this network structure is prese at the Fig. 5.

Table 1

The best forecast with Gaussian MF (inputs: 3; rules: 4; ratio: 0.6)

The next experiment was carried out with hybrid DL network with triangular MF. After experiment were found optimal parameters and structure of the hybrid network: 5 inputs, 3 MF and 0.8 ratio. After experiments the accuracy of hybrid networks with different MF was compared. The results are presented in the Fig. 6. At the next series experiments LSTM networks were investigated. The goal of experiments was to find the optimal parameters. The following parameters varied: 1.5 2 1 0.5 0

Real 399,15 399,12 397,97 396,82 396,62 number of inputs 3-5, ratio training/test 0.6, 07, 0.8. The optimal parameters values were found: n=3, ratio 0.7. After that the LSTM with optimal parameters was applied for training and forecasting. The results are presented in the Table 3. Flow charts of training and validation are presented in the Fig. 7.

In the next experiments the efficiency of the best models of hybrid DL network and LSTM was investigated and compared with different ratios training/test sample. The corresponding results for different in puts are presented in the Tables 4-6. Analyzing these results, we may conclude GMDHneo-fuzzy network has better forecasting accuracy at the interval period 5 than LSTM for various ratios. In the next experiments the forecasting efficiency of hybrid DL networks and LSTM at different forecasting intervals were investigated. In the Table 7 forecasting results of GMDH-neofuzzy network and LSTM at interval 7 days and in the Fig. 8 forecasting accuracy are presented.

As it follows from the presented results the hybrid neo-fuzzy network has the better accuracy than LSTM at the interval 7 days. At the succeeding experiments the forecasting accuracy of both

LSTM 4,074 1,378 2,023

LSTM 4,715 2,107 5,412

LSTM 1,961 4,628 2,152

LSTM 1,806 1,899 1,518 1,511 1,224 0,932 0,877 networks was explored at middle-term fore casting with interval 20 days. The accuracy by MAPE is presented in the Fig. 9. 4.3.

Comparative experiments of hybrid DL networks and LSTM

In the final experiments the forecasting accuracy of hybrid GMDH network (GMDH-nf) and LSTM were compared at different forecasting intervals (short-term and middle-term). The corresponding results by criterion MSE and MAPE are presented in the Table 8 and Table 9 correspondingly where the point means the size of test sample.

Period 3

Period 5

Period 7

Period 20

Period 3

Period 5

Period 7

Period 20 2.5

2 1.5

1 0.5 0

GMDH-nf

LSTM

GMDH-nf

LSTM

GMDH-nf

LSTM

GMDH-nf

LSTM

GMDH-nf

LSTM

GMDH-nf

LSTM

GMDH-nf

LSTM

GMDH-nf

LSTM

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

GMDH-Neo-fuzzy

LSTM

Analysis of these results shows that in a whole hybrid DL network has the better accuracy than LSTM at different short and middle forecasting intervals (3, 5, 7, 20 days).

5. Conclusion

6. References

References

1. In this paper the problem of forecasting at financial market with different forecasting intervals was considered (short-term and middle-term forecasting) . For its solution it was suggested to apply hybrid deep learning (DL) networks based on GMDH and LSTM networks .

2. The experimental investigations were performed at the problem of forecasting Emerging Markets Bond Total Return Index (EMBRI) at NASDAQ stock exchange at the period since January till August 2022 .

3. Optimization of parameters of LSTM and hybrid networks was performed during the experiments. The optimal structure of hybrid DL network was constructed using GMDH method .

4. The experimental investigations of optimized LSTM and hybrid networks were carried out at different forecasting intervals and their accuracy was compared. In result it was established that application of hybrid DL networks has much better accuracy than LSTM at the problems of short-term and middle-term forecasting at stock exchanges .

[1] S. Hochreiter and J. Schmidhuber , "Long short-term memory" , Neural Computation , vol. 9 , 1997 , pp. 1735 - 1780 . doi: 10 .1162/neco. 1997 . 9 .8.1735.

[2] B. Hammer , "On the approximation capability of recurrent neural networks" , Neurocomputing , vol. 31 , 1998 , pp. 107 - 123 . doi: 10 .1016/S0925- 2312 ( 99 ) 00174 - 5 .

[3] C. Olah , " Understanding lstm networks" , 2020 . URL: https://colah.github.io/posts/2015-08- Understanding-LSTMs/.

[4] A. Graves , "Generating sequences with recurrent neural networks" , CoRR , vol. abs/1308.0850 , 2013 . doi: 10 .48550/arXiv.1308.0850.

[5] A. Graves , Supervised Sequence Labelling with Recurrent Neural Networks , Verlag, Berlin, Heidelberg: Springer, 2012 . doi: 10 .1007/978-3- 642 -24797-2.

[6] S.S. Haykin , Neural networks: a comprehensive foundation , 2nd ed. Upper Saddle River, N.J : Prentice Hall, 1999 .

[7] Ossovsky S. Neural networks for information processing, transl . from Polish .-M. - Finance and Statistics, 2002 - 344p . (rus).

[8] Wang F . Neural Networks Genetic Algorithms and Fuzzy Logic for Forecasting // Proc. Intern. Conf. Advanced Trading Technologies . - New York, 1992 , pp. 504 - 532 .

[9] Yamakawa T. , Uchino E. , Miki T. , Kusanagi H . A neo-fuzzy neuron and its applications to system identification and prediction of the system behavior // Proc. 2nd Intеrn. Conf. Fuzzy Logic and Neural Networks «LIZUKA-92». - Lizuka , 1992 , pp. 477 - 483 .

[10] I. Goodfellow , Y. Bengio and A. Courville , Deep Learning, MIT PRESS, 2016 . URL: http://www.deeplearningbook.org.

[11] Yuriy Zaychenko , Yevgeniy Bodyanskiy, Oleksii Tyshchenko, Olena Boiko, Galib Hamidov . Hybrid GMDH-neuro-fuzzy system and its training scheme . Int. Journal Information theories and Applications , 2018 . vol. 24 , Number 2, pp. 156 - 172 .

[12] Yu . Zaychenko, Galib Hamidov . The Hybrid Deep Learning GMDH-neo-fuzzy Neural Network and Its Applications . Proceedings of 13-th IEEE Intern Conference Application of Information and Communication Technologies-AICT2019 . 23 - 25 October 2019 , Baku, pp. 72 - 77 .

[13] Evgeniy Bodyanskiy , Yuriy Zaychenko, Olena Boiko, Galib Hamidov, Anna Zelikman . Structure Optimization and Investigations of Hybrid GMDH-Neo-fuzzy Neural Networks in Forecasting Problems . System Analysis & Intelligent Computing. Ed. Michael Zgurovsky , Natalia Pankratova . Book Studfies in Computational Intelligence, SCI , vol. 1022 . Springer, 2022 , pp. 209 - 228 .