Application of Neural Network Technologies in the Analytical System "SHM Horizon"

Abstract

Strategic planning and management of the economics of Russian Federation and its regions requires an integrated approach to forecasting the key indicators of socio-economic development. This approach is implemented in the hybrid system of short-term forecasting of socio-economic indicators "SHM Horizon", developed by the authors. The implementation of the hybrid approach allows at the first stages to build econometric regression models of indicators, to verify forecasts and identify problem indicators, and, at subsequent stages, for a group of such indicators to build models based on neural networks and decision trees. The purpose of the article is to use the neural network module of the system in predicting the socio-economic indicators of the Russian Federation. In this study the forecasts were built for the main blocks of key indicators: macroeconomics, federal budget, socio-economic and indicators of foreign economic activity. The required quality and accuracy of forecasts was preset. Econometric models showed satisfactory results for 80% of indicators. For the other 20% of indicators calculations were carried out based on the neural network architecture of the multilayer perceptron. As a result, it was possible to achieve a significant improvement in the accuracy of forecasts for the vast majority of indicators and quality for half of them. Our approach and the developed system of hybrid models can be used for forecasting economic development both at the level of the Russian Federation and at regional and municipal levels.

Keywords

• Artificial Neural Networks
• Socio-Economic Indicators of the Russian Federation
• Forecasting
• Time Series
• Hybrid Information and Analytical System

-

An important task of strategic planning and management is to build a system of shortterm forecasting of socio-economic indicators. An econometric approach based on systems of regression equations has played a significant role before and remains relevant. However, the use of linear regression equations to predict economic performance has its limitations. First, there may be significantly non-linear relationships between indicators, which are difficult to describe on the basis of linear functions. Secondly, new indicators appear from time to time in statistical reporting for which there are no historical values (short series). Third, the values of indicators can be influenced by hidden factors, which cannot be taken into account explicitly.

At the same time, over the past 20 years, artificial neural networks (ANNs) have been successfully used to model nonlinear dependencies. Neural networks are widely applied to forecasting in different areas of economics and business. Time series forecasting based on neural networks plays an important role in these areas. At the same time, forecasting economic indicators cannot be qualitatively solved based on the analysis of individual time series and individual models. Here, an important role is played by a hybrid approach based on the construction of a forecasting system that implements regression, neural network and other models, and the choice of the desired model is based on the assessment of the quality and accuracy of forecasts. This approach is implemented in the system of predictive models that we are developing, the "System of Hybrid Models Horizon". “SHM Horizon” is intended for modeling and forecasting development indicators at the country and regional levels.

“SHM Horizon” implements a proven system of multilevel models of socioeconomic indicators based on interconnected regression equations (regression module). To predict indicators that are unsatisfactorily described by this system of models, a module for forecasting time series based on neural networks has been developed. The purpose of the article is to use the regression and neural network module of the system for predicting indicators of macroeconomics, state budgets, social sphere and foreign economic activity of the Russian Federation.

The article provides an overview of works on neural network methods, with special attention paid to architectures and forecasting models based on neural networks. 2

Literature Review

For the implementation of strategic planning and management, it is necessary to have a system for forecasting socio-economic indicators. Such systems are traditionally based on econometric models. In 1955, R.L. Klein applied an econometric approach to predict the macroeconomic indicators of the US economy [ 1 ]. Analytical reviews and a mathematical description of such models are given in the works [ 2 ] and [ 3 ]. The monograph [ 4 ] describes econometric country models and identifies directions for the design and development of macroeconomic forecasting systems. The econometric approach to forecasting was further developed in [ 5-7 ]. In the work [ 8 ] a range of econometric country models is discussed. Models for Russian economics is overviewed in [ 9 ].

The features of the time series of socio-economic indicators are relatively short data series, the presence of structural shifts and the relationship between different groups of indicators, as well as nonlinear dependencies between indicators. Therefore, linear models are not always able to give qualitative results. In this regard, a hybrid approach based on the construction of an ensemble of hybrid models is promising, in which, along with regression models, models based on neural networks, decision trees, and neuro-fuzzy networks are used. In recent years, in many works, the hybrid approach has been developed to forecast time series [ 10-14 ].

It should be noted that in Russia there is a number of systems for forecasting socioeconomic indicators, however, these are closed proprietary systems and are aimed at solving individual particular problems.

In Plekhanov Russian University of Economics, the authors are developing a specialized information and analytical system "SHM Horizon" (SHM - a system of hybrid models), which allows for building scenario based medium and short-term forecasts of more than 600 socio-economic indicators of the Russian Federation. It’s architecture, implemented models and methods are described in detail in [ 15, 16 ]. At the moment, the system comprises a module for regression models of socio-economic indicators of the Russian Federation, a neural networks module and a forecasting module based on decision trees.

Our study is devoted to predicting socio-economic series using both regression models and neural networks, therefore, further we give a review of works in the field of applying various ANN architectures in forecasting.

Artificial neural networks are used in a wide range of areas, including economics [ 17, 18, 25 ] and finance [ 19-21 ]. Research on time series forecasting based on neural networks has been developing for over 20 years. One of the first works on the methodology of using neural networks in forecasting was the work of Zhang, 1998 [ 22 ], which compares the results of forecasting based on a neural network with a number of statistical methods. The efficiency of time series modeling based on a multilayer perceptron in combination with the Back-propagation algorithm is demonstrated in the works of G. Peter Zhang [ 23 ], Michael Štencl 2011 [ 24 ].

Although the multilayer perceptron architecture has found wide application, recurrent networks are also used to predict economic series. The article [ 26 ] considers a number of neural network architectures, and compares the results of forecasting economic series (exchange rate, stock market index, economic growth indicator) based on different architectures.

The paper [ 27 ] describes the method of weighted online learning (WG-Learning) of the LSTM recurrent network for forecasting time series in the presence of outliers, tested on extensive experiments with both synthetic and real data sets. The work [ 28 ] proposes a new end-to-end architecture of a recurrent neural network based on an extended attention mechanism for modeling and forecasting time series of economic indicators containing missing values.

In [ 29 ], using the example of forecasting a macroeconomic indicator (monthly CPI in the USA), demonstrates that recurrent neural networks LSTM outperforms all other forecasting techniques including statistical models and architecture of a multilayer perceptron.

Application of models of recurrent neural networks - LSTM and GRU, for forecasting time series in the study [ 30 ] did not show the difference in forecasting efficiency of these 2 models. The authors of [ 31-32 ] note the superiority of LSTM network models over autoregression in predicting US macroeconomic indicators, which they associate with the nonlinearity of the indicators.

Based on the studies considered, it can be concluded that for forecasting economic time series based on ANN, it is advisable to use the multilayer perceptron architecture and, if it is necessary to take into account long-term dependencies, - recurrent LSTM networks. 3

Materials and methods

In the considered hybrid approach to forecasting economic indicators, two types of models are distinguished: balance-econometric systems of models and intelligent models based on artificial neural networks, decision trees, etc. (Fig. 1).

Based on a study of sources published by the Federal State Statistics Service, the Central Bank, the Ministry of Finance and the Ministry of Economic Development of the Russian Federation, groups of indicators were identified that characterize the state of the country's economic development. For modeling, both data published by the Federal State Statistics Service and those downloaded from the Contour BI system deployed in the situational center of the Plekhanov Russian University of Economics. The increment of the studied time series, as well as the forecast step, is equal to one year. The system contains data from 2003 to 2019.

Our work examines the time series characterizing the economic indicators of the Russian economy. The general structure of blocks of indicators is shown in Fig. 2. A block of foreign economic activity is currently under development.

In the system SHM Horizon system a scenario approach to forecasting is adopted. The development trajectory is set by a group of scenario indicators determined on the basis of a forecast of the long-term socio-economic development of Russia for the period up to 20301.

In the model, the following were chosen as scenario conditions:  The refinancing rate of the Central Bank  Rate of growth of money supply  Average export prices for Urals oil  Change in the international foreign exchange reserves of the Russian Federation  Gross domestic product

In the block for calculating quality indicators, the researcher can set the acceptable values for quality and accuracy indicators. Quality indicators include the coefficient of determination (R2), the Durbin-Watson test (DW), and the Fisher's F-test. The accuracy is estimated by the value of the mean relative error Δ (MAPE) based on the retro forecast.

The accepted limits for the values of the accuracy and quality criteria are shown in the Table 1. 1 Forecast of long-term socio-economic development of the Russian Federation for the period up to 2030 (developed by the Ministry of Economic Development of the Russian Federation), http://www.consultant.ru/document/cons_doc_LAW_144190, last accessed 2020/10/08.

The neural network module of the system is built on the basis of the architecture of the multilayer perceptron (MLP, Multy Layer Perceptron), which is a fully connected neural feedforwarded network [ 33 ].

The multilayer perceptron architecture assumes an input layer, an output layer, and one or more inner layers. On all layers, except for the input layer, a non-linear activation function is used for signal transmission. In a multilayer perceptron, the signal is transmitted in one direction, from left to right from layer to layer. Fully connected network means that each neuron in any layer of the network is connected with all neurons of the previous layer. In most cases, sigmoid and hyperbolic tangent are used as activation functions in MLP.

In 1989, Cybenko [ 34 ] proved that a feedforward artificial neural network with one hidden layer can approximate any continuous function of many variables with any accuracy, therefore MLPs are successfully used in the construction of regression models. However, when solving specific modeling problems, it is necessary to carry out multiple experiments to determine the network configuration, algorithm and training parameters in order to achieve the required model accuracy with an acceptable training time. Particular attention should be paid to the quality of the constructed model, i.e., its generalizing ability.

The most widely used algorithm for training a multilayer perceptron is the Back Propagation algorithm proposed in the works [ 35-37 ]. In the Back Propagation algorithm, the correction of the weights of neurons in the hidden layers of the neural network is calculated based on the output error of the network. The algorithm is simple to implement and allows you to train the network in a reasonable time.

However, since the Back Propagation algorithm uses the gradient descent method, which is one of the local optimization methods, the network can fall into a local minimum and no further training will be performed. Another disadvantage of MLP networks for forecasting is that the do not have memory, that is, they cannot process sequences of arbitrary length. In the case of time series, this means that it is impossible to take into account the influence of the previous states of the series on the current predicted value. In our case this is not a problem, since the studied series of indicators have a short length.

Results

In the system "SHM Horizon" modules have been developed for building, along with regression predictive models, intelligent models, including neural networks and regression decision trees. As our experiments with the calculation of the Russian Federation indicators for the social sphere show [ 15 ], the use of neural network models to predict a number of indicators for which the regression model gives unsatisfactory results made it possible to implement high-quality and accurate forecasts for the entire set of indicators. The process of building models in the system is shown in Fig. 3. The system is developed on the DotNet platform (.core) in the C # language. The "Neural Networks" module is implemented using the FANN library which was created in the С ++ language and was adapted for the С # language. The library has methods for building multilayer perceptrons and training the network using the backprop method. The method of creating a neural network allows you to describe the configuration of a neural network and set the activation function (relu, sigmoid, hyperbolic tangent, etc.). The network training method takes as a parameter the training rate, the number of epochs and the admissible error on the training and test samples. The initial weights are given by the weight matrix. The results of the method are training errors, calculation results at each stage of the neural network, and weights. The advantage of the FANN library is its efficiency and the availability of quality documentation. The library has an open source, which made it possible to use it within the framework of the development of the "SHM Horizon".

Within the framework of this study, calculations were carried out for the following blocks of indicators: macroeconomics, federal budget, socio-economic and indicators of foreign economic activity. Blocks and groups of indicators are shown in Table 2.

Macroeconomic Federal budget Socio-economic Foreign economic indicators indicators indicators activity  Federal budget  Cash income of  Export figures  Production of revenues as% of the population  Export by varigoods in GDP GDP billion rubles. ous types of  Service produc-  Corporate in-  Cash income of goods tion in GDP come tax the population  Import rates  Net (excluding  Value added tax by type (remu-  Import for varisubsidies) taxes  Income from neration of em- ous types of on products foreign econom- ployees, income goods  Remuneration ic activity (cu- from property, of employees 1 mulative) income from en(reporting data  Federal budget trepreneurial acof the Russian expenditures tivity, etc.) Federal State  Public admin-  Cash expendiStatistics Ser- istration tures of the vice GDP2)  Total FB ex- population  Gross profit of penses  Use of monetary the economy  Surplus (+), incomes of the (gross mixed in- deficit (-) of the population (purcoonme)pNroedtutcatxioens federal budget acnhadspeayomfegnot ofdosr and export- services, payimport opera- ment of mandations tory payments  Final consump- and contribu

tion expenditure tions, etc.)  Gross capital  Average per formation capita monetary income of the population  Population

growth rates  Labor market indicators

At the first stage, calculations were carried out for the econometric model: linear regression equations were constructed for all 60 indicators and verification was carried out. The verification results are presented in Table 3. 0,71 0,87 0,78 0,92 0,87 0,9 0,87 0,51 0,41 0,92 0,81 0,76

When forecasting using ANN, for 12 indicators (except for the indicator Number of dropouts) the forecast accuracy (MAPE) was increased significantly, and for three of them the forecast quality (R2) was significantly improved.

As an example of the success of the neural network model, retro forecasts using regression and neural network models for the indicator "Savings of the population as a percentage of income" are shown in Fig. 4 and Fig. 5.

Fact

Model

Discussion

Country econometric models continue to play an important role in planning and forecasting key indicators of socio-economic development. They allow to take into account the interdependencies of indicators associated with their economic content.

At the same time, many economic time series are characterized by nonlinear dependencies, and the influencing factors are often impossible to describe explicitly in the form of regression equations. This makes, as shown by numerous studies, the application of neural network methods and technologies very promising.

Over the past decade, approaches to forecasting time series based on feedforward networks and recurrent networks have been developing, including their application for forecasting economic indicators. At the same time, machine learning platforms and tools are becoming widespread, making it possible to implement all the main architectures and methods of training neural networks.

It should be noted that the use of neural network tools for separate unrelated time series does not allow for a systematic approach to modeling the economic sphere. We are convinced that a hybrid methodology should be applied with the possibility of choice in each case the most appropriate method. Such an approach is implemented in developed by the authors specialized hybrid forecasting system “SHM Horizon” system.

The results presented in the article demonstrate its success in forecasting the set of indicators of macroeconomics, state budgets, social sphere and foreign economic activity of the Russian Federation. 6

Conclusion

Calculations for predicting indicative indicators of the development of the Russian Federation based on regression and neural network models have been carried out. The presented study made it possible to obtain the following results:  a hybrid approach to building models and forecasts has been developed, in which, at the first stage, a regression model is built for all the studied indicators, then a multiple regression model is checked based on expert estimates of quality and accuracy values, and at the third stage, intelligent models are used for indicators with unsatisfactory values based on machine learning;  in the “SHM Horizon” system the forecasting of a system of 60 indicators of macroeconomics, state budgets, social sphere and foreign economic activity of the Russian Federation was performed using hybrid models;  for 47 of 60 indicators the regression model showed high and satisfactory values of quality and accuracy; for 13, the improvement in the quality and accuracy of the forecast was achieved through the use of a model of artificial neural networks based on the multilayer perceptron architecture.

The system "SHM Horizon" can find application in predicting the indicators of the development of regions of the Russian Federation, in particular in regional situational centers. Integration of the system with regional situational centers will make it possible to carry out more accurate target planning based on forecasts, as well as to predict the likelihood of crisis situations. 7

Acknowledgments

This study was carried out within the framework of the scientific project No. 20-5700024 "Development of models and technologies for assessing the state of components of large-scale socio-economic and organizational-technical systems based on artificial intelligence methods", which received support from the Russian Foundation for Basic Research as a result of the competitive selection of scientific projects as the winner of the competition Bel_a - Competition for the best projects of fundamental research, held jointly by the RFBR and the Belarusian Republican Foundation for Fundamental Research.

References

1. Klein , L.R. , Goldberger , A.S.: An Econometric Model of the United States, 1929 - 1952 . North-Holland Publishing Company, Amsterdam ( 1955 ).

2. Naylor , T.H.: Policy Simulation Experiments with Macroeconometric Models: The State of the Art . American Journal of Agricultural Economics . 52 ( 2 ), 263 - 271 ( 1970 ). https://doi.org/10.2307/1237498

3. Deistler , M. , Oberhofer , W. : Macroeconomic Systems . IFAC Proceedings Volumes . 5 ( 1-3 ), 436 - 450 ( 1972 ). https://doi.org/10.1016/S1474- 6670 ( 17 ) 68439 - 8

4. Shevelev , A. : Macroeconometric Model of the US . In: Proceedings of International Conference on Application of Information and Communication Technology and Statistics in Economy and Education (ICAICTSEE) , pp. 270 - 274 . UNWE, Sofia, Bulgaria ( 2014 ).

5. Marques , J. , Pascoal , R.: Mathematical Economics - Marginal analysis in the consumer behavior theory . In: Proceedings of the 19th SEFI-MWG European Seminar on Mathematics in Engineering Education ( 2018 ).

6. Johnston , J. , Dinardo , J.: Econometric Methods . 4th ed. New York: McGraw-Hill ( 1997 ).

7. Dougherty , C. : Introduction to Econometrics. 4th ed. Oxford University Press ( 2011 ).

8. Grishin , V.I. , Abdikeev , N.M. , Kolmakov , I.B. , Voronova , T.A. , Turlak , V.A. , Philippov D.I.: The system of account look-ahead indicators of macroeconomic of Russia . Financial Analytics: Science and Experience . 3 ( 13 ), 2 - 15 ( 2010 ). (in Russian)

9. Antipov , V.I. , Kolmakov , I.B. , Pashchenko , F.F. : The impact of compensation and productivity on the final consumption of households and GDP growth rates in the long term . Studies on Russian Economic Development . 18 ( 4 ), 403 - 416 ( 2007 ). https://doi.org/10.1134/S1075700707040065

10. Rutkovskaya , D. , Pilinsky , M. , Rutkovsky , L. : Neural networks, genetic algorithms and fuzzy systems . Goryachaya liniya, Telekom , Moscow ( 2006 ). (in Russian)

11. Averkin , A.N. , Titova , N.V. , Agrafonova , T.V. : Synthesis of Distributed Fuzzy Hierarchical Model in Decision Support Systems in Fuzzy Environment . In: Štěpnička, M. , Novák , V. , Bodenhofer , U . (eds.) New Dimensions in Fuzzy Logic and Related Technologies . Proceedings of the 5th EUSFLAT 2007 Conference 2007 , vol. 1 . Universitas Ostraviensis, pp. 377 - 379 . Ostrava, Czech Republic ( 2007 ).

12. Wang , J.S. , Ning , C.X. : ANFIS Based Time Series Prediction Method of Bank Cash Flow Optimized by Adaptive Population Activity PSO Algorithm . Information. 6 ( 3 ), 300 - 313 ( 2015 ). https://doi.org/10.3390/info6030300

13. Gunasekaran , M. , Ramaswami , K.S.: A Fusion Model Integrating ANFIS and Artificial Immune Algorithm for Forecasting Indian Stock Market . Journal of Applied Sciences . 11 ( 16 ), 3028 - 3033 ( 2011 ). https://doi.org/10.3923/jas. 2011 . 3028 .3033

14. Šebestová , M. : Bankruptcy Prediction for Manufacturing Companies Using the ANFIS Model . In: Sustainable Economic Development and Application of Innovation Management from Regional expansion to Global Growth , pp. 1071 - 1080 . International Business Information Management Association (IBIMA), Seville, Spain ( 2018 ).

15. Kitova , O. , Savinova , V. , Dyakonova , L. , Kitov , V. : Development of hybrid models and a system for forecasting the indicators of the Russian economy . Espacios . 40 ( 10 ), 18 - 24 ( 2019 ). http://www.revistaespacios.com/a19v40n10/19401018.html

16. Kitova , O.V. , Kolmakov , I.B. , Dyakonova , L.P. , Grishina , O.A. , Danko , T.P. , Sekerin , V.D.: Hybrid intelligent system of forecasting of the socio-economic development of the country . International Journal of Applied Business and Economic Research . 14 ( 9 ), 5755 - 5766 ( 2016 ). https://serialsjournals.com/abstract/95376_ch- 6 .pdf

17. Swanson , N.R. , White , H.: A Model Selection Approach to Real-Time Macroeconomic Forecasting Using Linear Models and Artificial Neural Networks . The Review of Economics and Statistics . 79 ( 4 ), 540 - 550 ( 1997 ). https://doi.org/10.1162/003465397557123

18. Chen , X. , Racine , J. , Swanson , N.R. : Semiparametric ARX neural-network models with an application to forecasting inflation . IEEE Transactions on Neural Networks . 12 ( 4 ), 674 - 683 ( 2001 ). https://doi.org/10.1109/72.935081

19. White , H. , Racine , J.: Statistical inference, the bootstrap, and neural-network modeling with application to foreign exchange rates . IEEE Transactions on Neural Networks . 12 ( 4 ), 657 - 673 ( 2001 ). https://doi.org/10.1109/72.935080

20. Hartford , J. , Lewis , G. , Leyton-Brown , K. , Taddy , M. : Counterfactual Prediction with Deep Instrumental Variables Networks . arXiv:1612.09596 ( 2016 ).

21. Heaton , J.B. , Polson , N.G. , Witte J.H. : Deep Learning in Finance . arXiv: 1602 .06561 ( 2016 ).

22. Zhang , G., Patuwo , B.E. , Hu , M.Y.: Forecasting with artificial neural networks: The state of the art . International Journal of Forecasting . 14 ( 1 ), 35 - 62 ( 1998 ). https://doi.org/10.1016/S0169-2070( 97 ) 00044 - 7

23. Zhang , G.P. , Qi , M. : Neural network forecasting for seasonal and trend time series . European Journal of Operational Research . 160 ( 2 ), 501 - 514 ( 2005 ). https://doi.org/10.1016/j.ejor. 2003 . 08 .037

24. Štencl , M. , Šťastný , J.: Artificial Neural Networks Numerical Forecasting of Economic Time Series . In: Hui, Chi-Leung (ed.) Artificial Neural Networks - Application , pp. 13 - 28 . InTech ( 2011 ). https://doi.org/10.5772/15341

25. Mvubu , M. , Kabuga , E. , Plitz , C. , Bah , B. , Becker , R. , Zimmermann , H.G. : On Error Correction Neural Networks for Economic Forecasting . In: 2020 IEEE 23rd International Conference on Information Fusion (FUSION) , pp. 1 - 8 . Rustenburg, South Africa ( 2020 ). https://doi.org/10.23919/FUSION45008. 2020 .9190244

26. Huang , W. , Lai , K.K. , Nakamori , Y. , Wang , S. , Yu , L. : Neural Networks in Finance and Economics Forecasting . International Journal of Information Technology & Decision Making . 06 ( 01 ), 113 - 140 ( 2007 ). https://doi.org/10.1142/S021962200700237X

27. Guo , T. , Xu , Z. , Yao , X. , Chen , H. , Aberer , K. , Funaya , K. : Robust Online Time Series Prediction with Recurrent Neural Networks . In: 2016 IEEE International Conference on Data Science and Advanced Analytics (DSAA) , pp. 816 - 825 . Montreal, QC, Canada ( 2016 ). https://doi.org/10.1109/DSAA. 2016 .92

28. Laptev , N. , Yosinski , J. , Li , E.L. , Smyl , S. : Time-series Extreme Event Forecasting with Neural Networks at Uber . In: International Conference on Machine Learning (ICML 2017 ). Time Series Workshop , pp. 1 - 5 . Sydney, Australia ( 2017 ). http://roseyu.com/time-seriesworkshop/submissions/TSW2017_paper_3.pdf

29. Almosova , A. , Andreseny , N.: Nonlinear Inflation Forecasting with Recurrent Neural Networks ( 2019 ).

30. Petnehazi , G.: Recurrent Neural Networks for Time Series Forecasting . arXiv: 1901 .00069v1 ( 2019 ).

31. Hardik , G. , Melnyk , I. , Oza , N. , Matthews , B. , Banerjee , A. : Multivariate Aviation Time Series Modeling: VARs vs . LSTMs ( 2016 ).

32. Verstyuk , S. : Modeling Multivariate Time Series in Economics: From Auto-Regressions to Recurrent Neural Networks ( 2020 ). http://dx.doi.org/10.2139/ssrn.3589337

33. Haykin , S. : Neural Networks: A Comprehensive Foundation . Subsequent edn . Prentice Hall ( 1998 ).

34. Cybenko , G.: Approximation by Superpositions of a Sigmoidal Function . Mathematics of Control, Signals, and Systems . 2 ( 4 ), 303 - 314 ( 1989 ). https://doi.org/10.1007/BF02551274

35. Werbos , P.J.: Beyond Regression: New Tools for Prediction and Analysis in the Behavioral Sciences . Ph.D. thesis , Harvard University, Cambridge, MA ( 1974 ).

36. Galushkin , A.I. : Analysis of Closed-Loop Multilayer Neural Networks . In: Neural Networks Theory , pp. 223 - 272 . Springer, Berlin, Heidelberg ( 2007 ). https://doi.org/10.1007/978-3- 540 - 48125-6_ 13

37. Rumelhart , D.E. , Hinton , G.E. , Williams , R.J. : Learning internal representations by error propagation. In: Parallel distributed processing: explorations in the microstructure of cognition , vol. 1 : foundations, pp. 318 - 362 . MIT Press, Cambridge, MA, USA ( 1986 ). https://dl.acm.org/doi/10.5555/104279.104293