TIME SERIES AND DATA ANALYSIS BASED ON HYBRID MODELS OF DEEP NEURAL NETWORKS AND NEURO- FUZZY NETWORKS

Abstract

Deep Learning has proven to be an effective method for making highly accurate predictions from complex data sources. In this paper we consider approach to data analysis and time series forecasting based on hybrid models. These models contain a Deep NN models and Neuro-Fuzzy networks. We are show an overview of new approaches for data science field - time series and data analysis. Also, we propose our models of DL and Neuro-Fuzzy Networks for this task. Finally, we show possibility of using these models for data science tasks. This paper presents also an overview of approaches for incorporating rule-based methodology into deep learning neural networks.

Keywords

• deep learning
• fuzzy models
• data science
• time series
• forecasting
• data analysis
• hybrid models

1. Introduction

Many machine learning tools can be effectively wielded by analysts to reduce the computational burden on an analyst [ 1 ]. One such machine learning tool, deep learning (DL), looks to encode complex mathematical representations using stacked auto encoders. One popular algorithm within DL is convolutional neural networks (CNNs), which have proven their utility in object classification [ 2 ] and detection [ 3 ] within imagery. CNNs can be used to process large amounts of data and sort out what and where an object is located without requiring analysts to manually sort through the imagery. While this is a useful tool, there exists a breakdown in communication between the operator and the CNN. The CNN is able to accurately generate a classification label but does not necessarily report on features that were present allowing a classification to be inferred. For example, a CNN may be able to correctly identify an object as being a ‘cat’ but not have any representation of ‘whiskers’ or ‘fur.’ Similarly, the analyst would not be able to communicate the importance of a specific feature or trait to the CNN which limits the amount and nature of feedback from an analyst to a CNN.

This problem fundamentally limits the utility of such tools. Without understanding how a CNN arrives at a solution, it is impossible to understand how adaptable the system is. This poses a complex challenge for many autonomous systems, as many of these machine learning tools are developed with controlled imagery and trained on labeled data. However, when an autonomous system is deployed the imagery may fundamentally change and there are no guarantees that the machine learning tools will operate effectively given these changes.

The paper deals with the task of building a hybrid time series forecasting system based on deep neural networks and cognitive modeling. This approach allows us to take into account both the quantitative and qualitative characteristics of the time series. For completeness, the features of fuzzy cognitive maps and their application in problems of time series forecasting are given. Also, the developed genetic algorithm for learning fuzzy cognitive maps is presented, which allows to avoid the time-consuming task of manually setting up a cognitive map. To solve the problem of working with semistructured data, which often take place in the tasks of forecasting time series, it is proposed to use deep neural network architectures, since such networks are able to operate with this type of data and show the most reliable results.

2. Deep neural networks in forecasting tasks

ANFIS is the abbreviation Adaptive Neuro-Fuzzy Inference System - an adaptive network of fuzzy output. Proposed in the early nineties [ 4 ], ANFIS is one of the first variants of hybrid neuralfuzzy networks - a neural network of direct signal propagation of a special type. The architecture of the neural-fuzzy network is isomorphic to the fuzzy knowledge base. Neuro-fuzzy networks use differentiated implementations of triangular norms (multiplication and probabilistic OR), as well as smooth functions. This allows the use of cross-fuzzy neural networks, rapid algorithms for learning neural networks, based on the method of back propagation of errors. The architecture and rules for each layer of the ANFIS network are described below. ANFIS implements the Sugeno fuzzy inference system in the form of a five-layer neural network of direct signal propagation [ 5 ].

The network inputs in a separate layer are not allocated. Figure 1 shows an example of an ANFIS network with two input variables (x1 and x2) and four fuzzy rules. In the example, the linguistic evaluation of the input variable x1, three terms are used, and for the variable x2 are used two terms.

3. Deep neural networks in forecasting tasks

Deep learning neural networks is based on teaching perceptions and not on specialized algorithms designed for specific tasks. Many deep learning methods were known as early as the 1980s, but the results were unimpressive [ 6 ], while advances were made in the theory of artificial neural networks (pre-training of neural networks using a special case of an undirected graphical model, the so-called limited Boltzmann machine) x (above all, Nvidia GPUs, and now Google’s tensor processors) did not allow creating complex technological architectures of neural networks with sufficient performance yu and allow to solve a wide range of tasks, do not be an effective solution before, for example, in computer vision, machine translation, speech recognition, with quality solutions, in many cases, are now comparable, and in some cases superior to "the protein" experts. Unlike machine learning, depth learning requires a much larger amount of training sample than in the case of machine learning. Also, unlike machine learning, a deep neural network can have thousands of layers. All this helps deep neural networks to achieve a sufficiently high accuracy in the tasks of analysis, classification and image recognition. But, the main drawback of in-depth training is the enormous resource-intensiveness; in order to train a deep neural network, it is sometimes necessary to make a training sample of a million images, or even more, and the learning process can take several days. For such tasks, even developed separate GPU processors to speed up the learning process.

To date, there are a number of libraries for deep neural networks. The most popular of them are Tensor Flow and Keras, which can be used in forecasting problems. In the work of scientists from Australia [ 7 ] Keras is used for short-term forecasting of energy consumption in the private sector. This task is very difficult, since the indicators in this case vary greatly. Scientists have concluded that prediction using a deep neural network allows to obtain acceptable results.

4. Hybrid system for time series analysis

The forecasting system is based on a modular architecture, which gives additional stability to the system, even if one of the modules fails, the other modules continue to do their work.

The system itself has three main modules responsible for the task of forecasting. The deep neural network performs a time series forecast based on numerical indicators and gives us a so-called quantitative forecast, the results of which pass through a verification system (assessment of forecast adequacy), if the forvecast corresponds to the required accuracy, it is transmitted to the next module. In parallel with the deep network, the module works with a fuzzy cognitive map, which receives input data on the eventual impact on the time series, a cognitive map is constructed, which takes into account all factors influencing a specific predicted indicator. At the output, a cognitive map gives us a forecast with the probability of its fulfillment, that is, with a consonance factor that tells us whether the forecast will be fulfilled or not. Further, all data obtained from these modules come to the third module, working on the basis of the neuro-fuzzy network, which aggregates the information obtained from the previous modules and gives the final forecast. In Figure 2 is a diagram of a forecasting system.

5. Acknowledgement

This work was supported by the Russian Foundation for Basic Research 17-07-01558.

6. Conclusion

In this paper, a hybrid model of a prediction system based on deep neural networks, fuzzy cognitive maps and the neuro-fuzzy network ANFIS was presented. This paper is an introduction for further research in relation to deep neural networks in forecasting tasks. As recent studies show, these neural networks have quite good prospects in these tasks, although at the moment the main and most popular field of application of deep neural networks is pattern recognition.

References

[1] Bonanno D. et al. An approach to explainable deep learning using fuzzy inference / /NextGeneration Analyst V. - International Society for Optics and Photonics , 2017 . - Т . 10207 . - С . 102070D .

[2] Krizhevsky , Alex, Ilya Sutskever , and Geoffrey E. Hinton . "Imagenet classification with deep convolutional neural networks . " Advances in neural information processing systems . ( 2012 ).

[3] Szegedy , Christian, Alexander Toshev , and Dumitru Erhan . "Deep neural networks for object detection . " Advances in Neural Information Processing Systems . ( 2013 ).

[4] J.-S. Jang , “ Anfis: adaptive-network-based fuzzy inference system , ”IEEE transactions on systems, man, and cybernetics , vol. 23 , no. 3 , pp. 665 - 685 , 1993 .

[5] D. Nauck , F. Klawonn , and R. Kruse , Foundations of neuro-fuzzy systems . John Wiley & Sons, Inc., 1997 .

[6] Yarushev S. A. , Averkin A. N. Review of studies on time series forecasting based on hybrid methods, neural networks and multiple regression //Программные продукты и системы . - 2016 . - № . 1 ( 113 ).

[7] Kong W. et al. Short-Term Residential Load Forecasting based on Resident Behaviour Learning //IEEE Transactions on Power Systems. - 2018 . - Т . 33 . - № . 1. - С . 1087 - 1088 .