In the field of time series predictions there are effective and consistent inaccuracies which potentially cause wrong economic decisions. Neural networks are seen as a potential option to improve predictions and are already used in some cases. Nevertheless it is unclear so far for which kind of time series data the use of neural networks is the right choice but also what kind of neural networks are feasible for which data characteristics. The main goal of this researchin-progress paper is therefore to conduct a between different time series characteristics and network architectures by using a descriptive analysis based on an extensive literature review. This goal is a step towards developing design guidelines for (deep) neural networks in the context of time series analysis.
The field of time series prediction through intelligent systems using Artificial Neural
Networks (ANN) is steadily growing [
Neural Networks are especially popular in the field of Big Data Analytics and its
subsequent domains like Natural Language Processing (NLP) and Pattern Recognition,
often times ranking first in prediction quality for several benchmark datasets [
When it comes to the task of time series prediction, neural networks have widely
been adopted for the task [
Our main goal within the full research project is to present guidelines on deep learning architectures based on time series data characteristics. Towards this goal we first aim to reveal the research gap of intransparent behavior of deep learning algorithms in the context of time series analysis. We therefore propose the following questions:
RQ1: What time series characteristics are usually present in the data when using neural network architectures?
RQ2: How do different architectures handle different characteristics? Our paper is structured as follows: We first give some preliminaries on time series characteristics in Section 2. After we described our research methodology in Section 3, we present the results of our descriptive analysis in Section 4 in order to answer research questions RQ1 and RQ2 in Sections 4.1 and 4.2 respectively. The paper closes with a discussion and an outlook in Section 5. 2
The most common characteristic in a time series is stationarity. We can assume a time
series as stationary when it has constant mean, variance and covariance for a given time
step regardless of time . When decomposing time series, we can yield a trend and a
seasonality component, if such effects are present in the current time series. If a time
series shows trends and seasonality they are not stationary, since they do not have
constant moments throughout the observation interval. Furthermore a time series can show
signs of non-linearity. Non-linearity can lead to small influences of big shocks and vice
versa. Influences in non-linear time series will also depend on the sign of the shock
effect [
As a basis for our descriptive analysis we employ a systematic literature review [
We initially found 226 papers. After we applied the technical constraints and removed duplicates 192 papers were left. We then filtered out papers with unsuitable non numeric time series data (170 papers left) and removed papers that were not topic relevant (e.g., no focus on intelligent systems) to our cause (135). In the next section we conduct a descriptive analysis and first answering RQ1 in Section 4.1 and subsequently RQ2 in Section 4.2
The majority of the found papers is published in the field of computer science. The research goals of the papers can be divided into model comparison, prediction (new problem), improve prediction (existing problem) and improve model. The improve prediction task makes up the majority of the papers (51%), followed by the improve model task (22%). The distribution of time series characteristics within the literature review database is depicted in Fig. 1. We can see that non-linear time series are the subject of a majority of papers along with trend and seasonal components. # of results 113 93 5 4 5 226
The different dataset sizes and their frequency of occurrence have a peak at around
500 data points, the majority of the studies does not extend into big data territory in
terms of volume. Only a small fraction of the studies use datasets with 10000
observations or more. This is particularly interesting when applying deep learning algorithms
that often times long for rather large training datasets. Considering the used algorithms
in the studies observed we used a taxonomy approach by [
Since in most cases from our literature review hybrid methods or special networks (no ANN) are used to model time series features we depict the correlation between hybrid modeling and no ANN with time series characteristics in Fig. 3. We clearly see that with most of the features like chaos or non-stationary present, hybrid models are far more often used, since standard ANNs are in some cases not able to achieve satisfactory results by themselves but rather need another additional method that either precedes or succeeds the application of the neural network. A special focus should also be given to the time architectural features of ANNs (e.g., LSTM, RNN) and their over proportional use when it comes to non-stationarity, chaos and especially noise. As in the classical models like GARCH or ARIMA, the noise is most likely autocorrelated and therefore has to be dealt with accordingly or otherwise it will be included into the model and will introduce a prediction bias. Considering the two advanced ANN classes “no ANN“ and “hybrid method”, it is apparent from the descriptive analysis, that some special treatment of time series data is needed and that the data cannot simply be processed by an out-of-the-box algorithm. When we look into the detailed algorithm classes of either class, we find a variety of different methods like Ridge Polynomial or Abductive Networks as well as fuzzy logic operators used as hybrid functions with classical ANNs that do not correlate with time series features in this small literature sample. Since time series data plays an increasingly large role in economic applications not only in the finance sector but also in the context of event and process as well as failure data in the context of Industry 4.0, the knowledge base is in need of guidelines on how to design networks in order to handle certain types of time series with different characteristics. Towards that notion, we conducted an extended literature review with regard to time series characteristics and neural network architecture. We showed that several of those characteristics are of great concern in the literature and that certain hybrid or nonbase ANNs are built for special cases. However, there are is no correlation between certain types of characteristics and those algorithms. This can be explained by the special designs of hybrid functions and networks, such as wavelet networks that are specifically designed and configured to deal with one particular problem. In general the transparency on how different time series characteristics can be dealt with in the context of neuronal networks remain foggy. As this work continues it will conduct extensive benchmarking studies based on simulated and real time events in order to find useful correlations between time series characteristics and neural network architecture in the context of prediction problems.