Traffic analysis systems are widely used in monitoring the network activity of users or a specific user and restricting client access to certain types of services (VPN, HTTPS) which makes content analysis impossible. Algorithms for classifying encrypted traffic and detecting VPN traffic are proposed. Three algorithms for constructing classifiers are considered - MLP, RFT and KNN. The proposed classifier demonstrates recognition accuracy on a test sample up to 80%. The MLP, RFT and KNN algorithms had almost identical performance in all experiments. It was also found that the proposed classifiers work better when the network traffic flows are generated using short values of the time parameter (timeout). The novelty lies in the development of network traffic analysis algorithms based on a neural network, differing in the method of selection, generation and selection of features, which allows to classify the existing traffic of protected connections of selected users according to a predetermined set of categories.
The term “deep package inspection” (DPI) [
In addition to analyzing network packets [
For identification, an analysis of the sequence of packets with the same characteristics is carried out. Analyzed characteristics are Source_IP: port - Destination_IP: port; packet size; frequency of opening new sessions per unit of time, etc. The analysis based on behavioral (heuristic) models corresponding to such applications.
The main component of the DPI solution [
Traffic analysis using traditional tools becomes impossible without selecting a key for streaming
data with encryption (for example, TLS / SSL protocols). It takes a lot of resources to find the key.
The relevance of hacking remains only at the governmental or military level [
In the [
In [
Therefore, the development of algorithms that allow classifying the traffic of secure connections
with the required level of detail by protocol is relevant [
Objective: improving the algorithms for analyzing the network traffic of secure connections. The main tasks of the study: 1. Analysis of algorithms for network traffic classification; 2. Development of the network traffic analysis system structure; 3. Development of the algorithm for analyzing the network traffic of secure connections on the basis of algorithms of feature generation and selection for construction a neural network classifier; 4. Software implementation of algorithms for analyzing network traffic and evaluating the effectiveness of the proposed solution on the basis of full-scale data.
Traffic classification [
There are two main methods of traffic classification (Figure 1): • Classification based on data blocks (Payload-Based Classification). It is based on the analysis of data packet fields. This method is the most common but does not work with encrypted and tunneled traffic. • Classification based on statistical analysis (time between packets, session time, etc.).
A universal approach to traffic classification based on information in the header of the IP packet.
This is usually IP address (Layer 3), MAC address (Layer 2) and the protocol used. This approach has
its limitations [
Deep package inspection (DPI) allows to implement more advanced classification . The main
mechanism for identifying applications in DPI is signature analysis [
There are several methods of signature analysis:
The applications contain certain sample sequences in the package data block. They can be used for identification and classification. Not every package contains a sample application data, so the method does not always work.
Numerical analysis uses the quantitative characteristics of the sequence of packets, such as: size of the data block, response time, interval between packets. Simultaneous analysis of multiple packets is time consuming, which reduces the effectiveness of this method.
Method is based on the analysis of traffic dynamics of the running application. While the
application is running, it creates traffic that can also be identified and labeled [
Protocols of some applications are a sequence of certain actions. Analysis of such sequences allows to accurately identify the application. reduces the effectiveness of this method.
Behavioral and heuristic analysis are used when working with encrypted traffic. For more accurate
identification, cluster analysis is used, which combines the methods of heuristic and behavioral
analysis [
The development of analysis algorithm for classifying network traffic of secure connections of
dedicated users according to a pre-defined set of categories is actual [
Two scenarios of network traffic analysis are considered: • analysis of encrypted traffic; • analysis of encrypted traffic passing through a virtual private network (VPN).
The figure 2 shows the organization's LAN structure with an analysis module for network traffic of encrypted connections.
Traffic arrives from the edge router. There is a seizure and preprocessing of traffic using the libpcap library. The primary features of the data flow are extracted from the received pcap files. Vector of primary features and sessions with a duration of 15, 30, 60 and 120 seconds is formed. The generation and selection of features for the neural network classifier training is performed. The prepared vector of features is fed to the neural network analysis module of user sessions. The settings for training and work are set by the administrator.
This process is shown in the figure 3.
Further, the following information comes to the decision block: the decision of the base block on the type of traffic, the probability of the traffic belonging to one of the basic types and the types of recognized traffic from the NA block analyzing user sessions. Administrator can perform the adjustment of the current solution on the decision block on the type of traffic.
Then, the current traffic from the decision block and marked traffic from the basic traffic analysis module (sender's IP, recipient's IP, port, sender, port of the recipient) are sent to the user sessions store.
Further, data on recognized user sessions is sent to the traffic type analysis module and the user type. An information security specialist receives information about the types of users and their rights. The administrator interacts with the repository to view and replenish the database and sets the parameters for capturing traffic.
Development of an algorithm for analyzing network traffic At the first step, the fragment of the intercepted traffic is downloaded, then the classifier scenario is selected. Based on the features indicated in the scenario, a training sample is formed to build the initial knowledge base. After analyzing the given features on the test sample, the accuracy of the classifier is determined. If the accuracy satisfies the requirements, the current state is saved, otherwise the cycle returns to the definition of the type of the script. The block diagram of the network traffic classifier is shown in Figure 4.
Traffic classification is based on the analysis of the temporal characteristics of the intercepted network packets stream for the formation of encrypted and VPN-traffic features (time-related features). The temporal characteristics of the flow make it possible to reduce the computational cost of building a set of features extracted from the encrypted network traffic by reducing the set of fixed parameters.
The experiment uses a dump of network traffic [
Function duration fiat biat flowiat active idle fb psec fp psec
Webbrowsing Email
P2P ICQ, AIM, Skype, Facebook, Hangouts Vimeo,
YouTube
and SFTP using FileZilla and external
service Voice calls on
Facebook,
Skype uTorrent and
transfer (BitTorrent)
The quality criterion for traffic classification is the accuracy of classifying samples. Assessment of
the classification accuracy can be carried out by cross-validation [
To solve the classification problem, the following algorithms are considered:
• Random Forest algorithm (RFT);
• K-Nearest Neighbor method (KNN);
• Multilayer Perceptron (MLP) [
As the source data, the real traffic generated by such applications and services as Skype, Facebook, etc. is used. Table 1 provides a complete list of the different types of traffic and applications included in the source dataset.
For each type of traffic (VoIP, P2P, etc.), open sessions and sessions are used in the created VPN tunnel, therefore there are in total 14 categories of traffic: VoIP, VPN-VoIP, P2P, VPN-P2P, etc.
Traffic was captured using the Wireshark sniffer. For VPN-traffic, the external service of the VPN provider was used, the connection was made using OpenVPN. To generate SFTP and FTPS traffic, use an external service provider and FileZilla as the client.
Let's define two different scenarios A and B (Figure 5). Four different time duration values were used to generate the data sets.
Scenario A: The purpose is to select the features of encrypted traffic with VPN identification, for example, distinguishing between voice calls (VoIP) and voice calls passing through VPN (VPNVoIP). As a result, there are 14 different types of traffic: 7 regular types of encrypted traffic and 7 types of traffic passing through the VPN. The first classifier uses VPN and non-VPN traffic separation, and then each traffic type classified separately (VPN and non-VPN).
The label identifies the application traffic for sending or receiving files and documents. The files were transferred to Skype, FTP through SSH (SFTP) and FTP through SSL (FTPS).
Scenario B: in this case we use a mixed data set. The classifier's input is regular encrypted traffic and VPN traffic, and the output is allocated the same 14 different categories.
Using general definition of traffic, it is determined by the sequence of packets with the same values for: destination IP address, sender's IP address, sender's port, sender's port, and protocols (TCP or UDP).
Streams are considered bidirectional. Together with the generation of traffic, the features associated
with each type of traffic are determined. TCP stream are usually terminated when the connection is
broken (by the FIN packet), and the UDP stream are terminated by a thread interruption. The value of
the thread interrupt can be assigned arbitrarily. In particular, we set the duration of the streams to 15,
30, 60 and 120 seconds [
MLP is a neural network of direct propagation. It consists of two hidden layers. In the first hidden
layer there are 30 neurons, and in the second there are 15 neurons. Size of input vector of features is
23. The activation function of neurons is a hyperbolic tangent [
When testing the KNN algorithm, the neighbor number parameter is 50.
When testing the random tree method, the classifier type “Bag” is used. The number of ensemble training cycles is 150.
The cross validation (Tables 4, 5) was performed in order to assess how classifiers are able to work with real data, while producing a result whose accuracy is correlated to accuracy in the test sample [36].
There is a direct relationship between the length of the captured session of the thread and the performance of the classifiers. When using the RFT classifier, the accuracy on the test sample decreases from 0.857 with a flow time of 15 seconds to 0.828 using a 120 second flow. Similar behavior is observed for the KNN and MLP algorithms. The best results are achieved using the RFT algorithm, with the time required to create the stream equal to 15. These results show that, using shorter time-out values for the traffic classifier, you can in-crease the accuracy value.
The second part of scenario A focuses on the separation of VPN and non-VPN traffic. The input is classified according to traffic categories. The results for shorter duration values are better than the results for larger values, albeit with some exceptions. In the case of VPN classifier, as VPN-mail, where the best result is obtained with the value of ftm equal to 30 seconds. In the case of a non-VPN classifier, the same thing happens.
All encrypted streams and VPN traffic are mixed in one set of data. The goal is to classify traffic without prior VPN separation from non-VPN traffic. There are 14 types of traffic: 7 encrypted and 7 VPN traffic types.
The short duration of the session of the captured stream does not provide the greatest accuracy. For example, for the MLP algorithm, the test accuracy is 0.795 and 0.51 for 15 seconds, and for a session time of 30 seconds, the accuracy on the test sample for the same algorithm is 0.798 and 0.637. The highest accuracy on the test sample for different interrupt values is 0.847 (RFT algorithm with a flow time of 120 seconds). 0.882
The cross-validation (Table 8) was performed to assess the effectiveness of the KNN method and the RFT method.
Traffic analysis systems are widely used in monitoring the network activity of users or a specific user and restrict the client's access to certain types of services (VPN, HTTPS). This makes analysis of content impossible. Algorithms for classification of encrypted traffic and detection of VPN traffic are proposed. Three algorithms for constructing classifiers: MLP, RFT and KNN, are considered.
The effect of the session length of the captured data stream on the accuracy of the classification is established. The developed classifier demonstrates the accuracy of recognition on the test sample to 80%. Algorithms MLP, RFT and KNN had almost identical indicators in all experiments.
It is also established that the proposed classifiers work better when network traffic flows are generated using short time-out values.
The novelty lies in the development of algorithms for analyzing network traffic on the basis of a neural network. This method differs in the way of features generation and selection, which allows classifying the existing traffic of protected connections of selected users according to a predefined set of categories.
The developed algorithms can improve the security of the data transmission network by improving the algorithms for analyzing network traffic as part of a data leak prevention system.
This work is partially supported by the Russian Science Foundation under grants № 17-07-00351.