Network disruptions cause significant financial losses and discomfort of customers. However, communication systems provide various data about equipment condition. This information can be used to predict network disruptions. Purpose of research is applying neural networks to network disruption prediction. Introduced approach has good feature extraction ability and data corruption resilience. Representation for network disruption dataset has been developed. Chosen network type is deep belief networks. Net structure variations have been proposed for chosen data representation and neural network type. Experimental research of proposed methods gives meager results for selected dataset. Results can be improved by net structure complication and by increasing dataset volume.
Type id fault_severity location event_type resource_type severity_type log_feature integer, unordered integer, ordered enumerated, unordered enumerated, unordered enumerated, unordered enumerated, unordered tuple: type – volume type: enumeration, unordered volume: integer, ordered
Repetition, values/event 12468/7381 8460/7381 23851/7381
Description Identifier, unique. Intended for data
union.
Class of event, forecast objective.
Network equipment location.
Type of event.
Type of resource that caused event.
Type of log message Features extracted from logs with its volume
As we can see, all significant attributes belong to enumerated (categorical) unordered type. This attributes shows record affiliation with some of class by some of parameter. For example, “location 1”, “event_type 15”, “resource_type 8”, “severity_type 2”. log_feature attribute is a tuple “type of value, volume of value” (e.g. “feature 35, 17”).
Attributes are kept in separate tables distributed by different CSV-files. This manner of data storage is used because of attribute values repetition. For example, single event can be of “event_type 15” and “event_type 11” at the same time. id attribute is used for table matching. Table structure is represented in figure 1 a. = ( 1, . . , ). = { 1, 0, ℎ ∈
. = { ′ ,
= .
ℎ Here is the amount of i-th attribute values; = ̅1̅̅,̅̅̅,
– amount of categorical attributes; is as follows: Here are values of i-th attribute for current event; – value of i-th attribute corresponding to j-th index. fault_severity input vector part constructs in the same way.
Values for log_feature are different from values for other attributes. They not only appeared or not appeared for current event but have volume. Corresponding elements of input vector possess this volume (normed) instead of “1” if present. Thus, last equation is as follows for log_feature attribute:
Here ′ is volume of k-th appeared feature; – -th appeared feature («type» in tuple); – value of log_feature corresponding to -th index, = ̅1̅,̅̅̅̅; – possible log_feature values amount; где – maximum value for j-th feature.
Thus, input vector V is a sequence of attribute vector parts , normed and has length N:
N is 1324 for “Telstra Network Disruptions” dataset. This is a large value. So, deep neural networks have been used.
Dee belief networks [3] have been chosen. This type of deep neural networks has good hidden feature extraction ability and can be fast trained and fine-tuned for high-dimensional datasets. Deep belief networks (DBN) is a composition of restricted Boltzmann machines (RBM) [4]. RBMs stacks layer by layer to construct DBN (figure 2).
Boltzmann machine (figure 3 a) is a stochastic machine formed by stochastic neurons [5]. This net fully connected. Neurons are divided into visible and hidden. Visible neurons are clumped onto values from dataset during the training. Hidden neurons always operate freely. These neurons can capture high order statistical correlations in the clumped values [5].
Boltzmann machine has one significant disadvantage: too long time of training. It caused by multiple neuron activation. Restricted Boltzmann machine avoid this problem by removing connections between neurons with the same type (figure 3 b). RBMs can be fast trained at practice [4].
Algorithm [6] of training DBNs are shown below (figure 2): 1) Train first two layers (h1, h2) as RBM using dataset. 2) Pass dataset through trained RBM and get values from layer “h2”. Values got from “h2”layer are a new dataset. 3) Train next pair of layers (h2, h3) using new dataset, then repeat step 2 and 3 for next layers and so on until last hidden layer trained. Previous steps are called “pretraining phase”. 4) Train whole net using backpropogation or another common algorithm to fine-tune weights. This step is called “training phase” Set of possible architecture variations forms during research: Various types of nets (DBN, Perceptron) Different layers count Various layer sizes Neural nets have 1324 neurons at input layer and 3 neurons at output layer for “Telstra Network Disruptions” dataset. This dataset can be divided into train and test parts using three different ways: 1) Random division. 2) Division for each location. Events for each location are divided into train and test parts separately.
3) Division by location. All events for certain location occur in one of sets entirely.
Regular accuracy ranking function has been used. It is a right-to-total ratio:
P A = .
N Here A is accuracy; P is a count of correct predictions; N – total predictions count.
Two loss functions have been tried: mean square error and weighted mean square error.
I imbalance often causes degenerate models construction. These models classify any input vector into one and the same class, the largest class.
Described problem can be solved via using a weighted mean square loss function: ( ) = ∑(( ̂ − )) × .
2 Here ( ) – weighted mean square loss function, – significance of prediction error for i-th class.
Setting inversely proportional to class occurrences number helps to solve the problem.
Best accuracy values for different networks consist of two and three hidden layers are listed in table 2.
1
dataset experiments.
Two types of networks have been considered: deep belief networks and perceptron (as a comparison). Accuracy and loss function have been measured via train or test selection. Additional data from test part of source dataset has been used in one of Deep belief networks reach better results in comparison with perceptron for selected dataset and train vector representation. Accuracy is increasing with the growth of hidden layers count. Therefore, it is reasonable to try networks that are more Additional data usage causes accuracy growth. It shows that better results can be reached for bigger datasets.
There is an interesting approach to use neural networks for telecommunication disruptions prediction. It is reasonable because of high dimension of input data. Deep belief networks have been selected. This type of deep neural networks has good hidden feature extraction ability and can be fast trained and fine-tuned for high- dimensional datasets.
Best reached accuracy is 75.7 % of correct predictions. This result can be improved by net structure complication. Bigger dataset usage also can help. [1] Li H, Li XY, Ramanathan M. Identifying informative risk factors and predicting bone disease progression via deep belief networks. Methods 2014; 69(3):