Internet traffic forecasting models with learning ability, such as the artificial neural network (ANN), have been growing in popularity in recent time due to their impressive performance in modelling the high degree of variability and nonlinearity of internet traffic. This study examined the impacts of some design issues on performance of the multi-layer artificial neural network for internet traffic forecasting. The traffic forecasting was modelled as a standard time series problem and the multilayer artificial neural network designed to performs the time series function mapping. The input lags were varied from 1 to 24. The training epoch values of 200, 500, and 1000 on one and two hidden layered networks were used. The learning algorithm was backpropagation with 0.1 learning rate and 0.9 momentum on logistic sigmoid activation function. The model was implemented in Visual Basic and validated with four categories of classified time series internet traffic of a branch residential network of one of a firm in Nigeria. Various predictive performances without consistent pattern were observed on the issues considered, however, input lag one gave the worst performance in all cases for the HOURLY traffic; three of the four traffic categories demonstrated the superiority of two hidden layers to one hidden layer. Although the epoch values of 200, 500 and 1000 showed no consistent performance variations, epoch value 200 outperformed the others on the model selections. The study revealed that input lags, number of hidden layers and epoch values could impact on the traffic forecasting performance of multilayer perceptron and that performance could be considerably improved by careful selection of those parameters through experimentations.
Accurate information about offered traffic is required for efficient resource provisioning and general capacity planning of an Internet service. The inability of most statistical methods in modelling the high variability of internet traffic accurately, and their lack of reasoning capabilities have triggered an increased number of studies that employ non-traditional statistical methods including machine learning. Furthermore, traditional summary statistics, particularly the sample mean and variance are instable metrics for working with the high variability of internet traffic, as such the sample means and sample variances are not reliable statistics for summarising traffic properties [1]. Machine learning techniques, such as the Artificial Neural Network (ANN) employ mechanisms that allow computers to evolve behaviour based on knowledge gained from dynamic observations. Machine learning technique based on nonlinear elements is often referred to as Neural Network. Neural networks are networks of nonlinear elements interconnected through adjustable weights and they play a prominent role in machine learning. Artificial (ANN) emerged with the aim of imitating the information processing process of human brain. Through learning, ANNs can determine nonlinear relationship in a data set by associating the corresponding output to input patterns. The multilayer artificial neural network, among other machine learning models, has shown impressive results in forecating studies [2,3,4,5,6,7,8,9]. However, applying an ANN to a given forecasting endeavour is a hard task, as basic modelling issues must be carefully considered for enhanced precision. The issues include the network architecture, learning parameters and data pre-processing methods [6,8]). The inconsistencies in performance reports on the design issues in the literature was noted also in [8]. In [9] it was argued that ANN technique should not be applied arbitrarily as has been sometimes suggested and even used in the internet forecasting domain [10,11] .
The paper examined the impacts of number of input lags, hidden neurons and training epochs on the precision of the multilayer artificial neural network in forecasting internet traffic.
Quite a number of research efforts has been reported in the literature on seeking appropriate models for forecasting Internet traffic.
In [12] a comparative study on suitable statistical methods for network traffic estimation was conducted. In the paper, several estimation methods for IP network traffic were studied. The study showed that non-linear time series models could model and forecast better than the classical linear time series models. Anand in [13] investigated a non-linear Time series model, the Generalised Autoregressive Heteroskdasticity (GARCH) in internet traffic modelling. The model showed that the forecasting algorithm was accurate compared with actual traffic. Although nonlinear statistical models can capture the busrtiness of network"s traffic, the models are parametric in nature and therefore require the knowledge of the distribution of the traffic. In addition, they are analytical and therefore require explicit programming to clearly specify the algorithmic steps. To take the advantages of machine learning paradigms, applying machine learning techniques to internet traffic forecasting has been on the increase.
A vast number of research efforts have been on going in exploring machine learning techniques to internet traffic predictions, the results of which have demonstrated their superiority to statistical forecasting methods. A concurrent neuro-fuzzy model to discover and analyse useful knowledge from available Web log data was proposed in [14]. The study used self-organizing map for pattern analysis and a fuzzy inference system to capture the chaotic trend to provide short term (hourly) and long term (daily) web traffic trend predictions. Empirical results demonstrated that the proposed approach was efficient for mining and predicting web traffic. A study in [15] presented a neural network ensemble (NNE) for the prediction of TCP/IP traffic using time series forecasting (TSF) point of view. The NNE approach was compared with TSF methods (Holt -winter and ARIMA) and the NNE was found to compete favourably with the TSF methods. In [16] the least square support vector machines was applied to solve the problem of accurately predicting non-peak traffic and the method had a good generalization ability and guaranteed global minima. [17] Presented a neural network ensemble approach and two adapted time series methods (ARIMA and Holt-Winters) for forecasting the amount of traffic in TCP/IP based networks. The experiments with the neural ensemble achieved the best results for 5 min and hourly data, while the Holt-Winters was the best option for the daily forecasts. The study in [10] investigated the ensembles of artificial neural networks in predicting long-term internet traffic. The proposed prediction models were compared with the classic method of Holt-Winters. Prangchumpol in [18] presented a description approach to predicting incoming and outgoing data rate in network system by using a data (machine learning) mining techniques, the association rule discover. The result of the study showed that the technique could predict future network traffic.
A detailed state of the art presentation on forecasting with artificial neural networks was made in [8]. The study showed that overall; ANNs gave satisfactory performance in forecasting, but went on to indicate the inconsistencies in performance reports of design issues in the literature. The inconsistencies were attributed to trial and error methodology adopted in most studies. Faraway and Chatfield [9] argued that it was unwise to apply ANN models blindly in black box mode as had sometimes been suggested. Shamsuddin,et al. in [7] investigated the effect of applying different number of input nodes, activation functions and preprocessing techniques on the performance of backpropagation network in time series revenue forecasting. The findings showed that the performance of ANN model could be considerably improved by careful selection of those parameters. In [19], the performance of two learning algorithms: the linear regression and Neural Network Standard Back propagation were compared on the prediction of four major stock market indexes. The comparison showed that the neural network approach resulted in better prediction accuracy than the Linear Regression model. Chabaa et al in [20] presented an ANN based on the multi-layer perceptron for analysing a time series measured internet traffic data over IP networks. The comparison between some training algorithms demonstrated the efficiency and accuracy of the Levenberg Maquardt and the Resilient back propagation algorithms. Chukwuchekwa in [21] compared the performance of the back propagation gradient descent technique and genetic algorithm on some pattern recognition problems. The backpropagation (BP) algorithm was found to outperform the genetic algorithm in that instance. The study suggested that caution should be applied before using other algorithms as substitutes for the BP algorithm, more especially in classification problems. In [2], an evaluation of several learning rules for adjusting ANN weights was carried out on the popular airline passenger data set. The Levenberg-Marquardt backpropagation algorithm showed the best performance among other learning rule. Various degrees of performances were observed in [22] on examining the impact of input lags of the multilayer perceptron in forecasting internet traffic on a two layered network. In [23] a survey of research and application issues on Web usage mining based on various mining technique was conducted to provide some understanding in designing algorithms suitable for mining data.
This review demonstrated the impressive results of applying machine learning technique, such as the artificial neural networks, in forecasting Internet traffic as well as raising concerns over the little or no consideration given by researchers on the design issues. The paper therefore presents results from the study on the impacts of some multi-layer perceptron design issues on internet traffic forecasting.
The traffic forecasting was modelled as a standard time series problem and the multilayer artificial neural network designed to performs the time series function mapping.
Traffic forecasting is a standard time series prediction task. The goal is to approximate the function that relates the future values of a variable of the previous observations of that variable [24]. In some situations, such as internet traffic, data are non-stationary and chaotic. In such situation, one general assumption is that historical data incorporate all behaviour required to capture the dependency between the future traffic and that of the past. Therefore, the historical data is the major player in the forecasting process. The second assumption to model and forecast the dynamic of the traffic is that its values are expressed by discrete time series [2,3]. A discrete time series is a vector {y t } of observations made at regular intervals, t=1, 2, 3……, N. For the time series forecasting problem, the inputs are typically the past observation of the data series and the output is the future value. , where each output would have separate weights for each connection to the neurons. Secondly, "feedback" the one-step-ahead forecast to replace the lag 1 value as one of the input variables, and the same architecture could then be used to construct the two-step-ahead forecast, and so on [16].. This study adopted the latter iterative approach because of its numerical simplicity and because it requires fewer weights to be estimated.
Neural network is a powerful model for solving complex problems because it has natural potential of solving nonlinear problems and can esily achieve the input-out mapping, it is good for solving predicting problems [26]. The basic features of the multilayer perceptrons include:
i.
The model of each neuron in the network includes a nonlinear activation function that is differentiable. ii.
The network contains one or more layers that are hidden from both input and output nodes.
iii.
The network exhibits a high degree of connectivity, the extent of which is determined by synaptic weights of the network.
Figure . 1. Architecture of Multilayer Perceptron with two hidden layers
The node is the basic unit of the Artificial Neural Network. . Each node is able to sum many inputs x 1 , x 2 , …,x n form the environment or from other nodes, with each input modified by an adjusted node weight (Figure 2). The sum of these weighted inputs is added to an adjustable threshold for the node and then passed through a modifying (activation) function that determines the final output.
The neural model in Figure 2 includes an externally applied bias, denoted by k b . The bias has the effect of increasing or lowering the net effect of the activation function, depending on whether it is positive or negative, respectively. Mathematically, we may describe the neuron k depicted in Fig. 2 by the following equations:
where
The bias k b is an external parameter of neuron k.
The activation function, denoted by (v) defines the output of a neuron in terms of induced local field v. It is this function (also called, the transfer function) that determines the relationship between inputs and outputs of a node and a network. In general, the activation function introduces a degree of nonlinearity that is valuable for most ANN applications. Among these functions, sigmoid function is very popular. It is a strictly increasing function that exhibits a graceful balance between linear and nonlinear behaviour. The Logistic Sigmoid is defined as in ( 5)
A logistic sigmoid function assumes a continuous range of values from 0 to 1. Additional types of activation functions can be found in [8]. Among these functions, logistic transfer function is the most popular choice [8].
Training of artificial neural networks ANN has to be trained before it can be put to use. The goal of the training is to find the logical relationship from the given input/output. There two strategies of the learning: supervised and unsupervised. This study employs the supervised learning strategy. Supervised learning typically operates in two phasestraining and test set. The training set is used for estimating the arc weights while the test set is used for measuring the generalization ability of the network. Training is used to gain generalised knowledge about the system under consideration and testing is used to predict (forecast) the system behaviour using the knowledge gained. On the other hand, unsupervised techniques such as the reinforcement learning is independent of training data and operate by directly interacting with the environment.
The training algorithm employed is the Backpropagation. It is a supervised training strategy and popular method for training the multilayer perceptron. The training proceeds in two phases [26]:
1. In the forward phase, the synaptic weights of the network are fixed and the input signal is propagated through the network, layer by layer, until it reaches the output. Thus, in this phase, changes are confined to the activation potentials and outputs of the neurons in the network. 2. In the backward phase, an error signal is produced by computing the output of the network with desired response. The resulting error signal is propagated through the network, again layer by layer, but this time the propagation is performed in the backward direction. In this second phase, successive adjustment is made to the synaptic weights of the network.
In [5] it is also reported that the backprobagation is the most computationally straightforward algorithm for training the multi-layer perceptron. They summarized the algorithms steps as
Internet traffic data was collected in hourly average kilo bit/s of TCP/IP traffic of a company"s resident network from January 1 2010 to September 30 2010 (making up 6552 data points each for IN and OUT traffic data), daily traffic data from January 1 to December 31, 2010 (making up 365 data points each for IN and OUT traffic data), using PRTG (Paessler Router Traffic Grapher), a network monitoring and bandwidth usage tool from a company called PAESSLER. 20Mpbs bandwidth was allocated for upload (Traffic IN) and 20Mbps for download (traffic out) statically for the period under consideration.
Nonlinear activation functions such as the logistic function typically restricts the possible output from a node to, typically, (0, 1) or (-1, 1). This is to avoid computational problems, to meet algorithm requirement and to facilitate network learning. Four methods for input normalization are summarized in [8]. This study employs, the Linear transformation to [0, 1], defined as y n = (y 0 -y min )/(y max -y min )
where y n and y o represent the normalized and original data: y min , y max, are the minimum, maximum of the column or rows respectively.
Eighty percent (80%) of the data, that is, 5241.6 approximated to 5242 was used for training the network, while twenty per cent (20%), that is, 1310.4, approximated to 1310, was used for testing the generalisation predictive capability of the network each for the HOURLY_IN and HOURLY_OUT flow traffic. Also, a training set of 80% and testing set of 20% were used for each of the DAILY traffic, that is. 292 data points for training and 73 for testing. .
For times series forecasting problem, a training pattern consists of a fixed number of lagged observations of the series [7]. The inputs (number of lag observations) were varied from 1 to 24, excluding the bias. One and two hidden layers were considered. The number of hidden nodes were equalled to the number of input nodes. In several studies, networks with the number of hidden nodes being equal to the number of input nodes are reported to have better forecasting [8]. One output node was used, one look-ahead. So the model of our network is k, k, k, 1, where k represents the number of lag observations (input variables). The epochs were based on 200, 500, and 1000. The best model according to [18] is the one that gives the best result in the test set. The logistic sigmoid activation function was used [8]. The Error correction backpropagation algorithm with learning rate: 0.1; momentum: 0.9 was used to train the network.
The training stop after each epoch respectively. Typically, as SSE based objective function or cost function to be minimized during the training process is defined in (10). The measure of accuracy employed is the Root Mean Square error (RMSE) defined as where n is the total number of sample group observations, ŷt is the predicted (computed) value while yt is the target value at time t. RMSE is one of the most commonly used measure of forecast error to examine how close the forecast is to the actual value [5]. The best model is the one that gives the best result in the test set, that is, the model that has the least RMSE in the testing set [27]. 4.
The system was implemented in visual basic. The RMSE of various models were recorded and compared based on the design issues considered. The results are presented and discussed in this section.
The RMSE of the testing (prediction) results of the various models based on the number of input lags, number of hidden layers and training epochs on one and two hidden layers network respectively were compared for the HOURLY_IN traffic. Figure 3 depicts these results.
There were varying degrees of performance with no regular patterns of performance among the input lags, between the one and two hidden layers networks, and among the various epochs used. Nevertheless, the worst performance for all the cases is input lag 1. The least RMSE with the value 0.0766984 of this experiment occurred at input lag 24 with 200 training epochs on two hidden layer network. Therefore, the best model for forecasting the HOURLY_IN traffic is input lag 24, 200 training epochs using two hidden layers.
The RMSE of the testing (prediction) results of the various models were compared for the HOURLY_OUT traffic. The results are shown in Figure 4. There also various values of the performance measure with no particular patterns on the issues for the HOURLY_OUT traffic. As in the HOURLY_IN, the worst performance for was recorded at input lag 1 in all the cases. The least RMSE with the value 0.0621992 occurred at input lag 13 with 200 training epochs on two hidden layer network. Therefore, the best model for forecasting the HOURLY_OUT traffic is input lag 13, 200 training epochs using two hidden layers.
Figure 5 presents the prediction RMSE of the various models for the DAILY_IN traffic.
The DAILY_OUT traffic. 7 compares the predicted models selected for the traffic categories. The HOURLY traffic categories had a better prediction performance than the DAILY traffic counterparts. This could have been attributed to the very large sample size used for the HOURLY traffic. It has been reported that the ANN for forecasting perform better with large sample size than with small sample size (Zhang et al. 1998 [8] and Zhang, et al. [26]). In addition, figure 7 revealed that various forecasting models may exist for different traffic categories, even if the traffic categories are all from the same network operator. This study has observed different forecasting models for the various traffic categories based on the issues. The findings suggest that carefully consideration of the design issues is indispensable for improving the predictive performance of a multi-layer artificial neural network rather than applying it to internet traffic forecasting blindly. However, there are no generally acceptable techniques for determining the optimal design parameter but by experimentations, an improved predictive performance model is feasible.
This study examined the impacts of some important design issues in modelling a multilayer perceptron artificial neural network for Internet traffic forecasting. The traffic forecasting was modelled as a standard time series problem and a multilayer artificial neural network designed to performs the time series function mapping. The mechanism was implemented in a Visual Basic programming environment and tested with real Internet traffic data through experimentation with the various design issues considered. Although no particular pattern of performance was observed the study showed that the forecasting performance can be affected by the number of input lags, hidden layers and training epochs,. Despite that the study did not make any attempt to determine an optimal values for the various factors considered, it has shown that careful experimentation is required to choose appropriate values for each of the design issues. Therefore, the multilayer perceptron should not be applied blindly to Internet traffic forecasting.