Intrusion Detection Systems (IDSs) play a key role in modern ICT security. Attacks detected and reported by IDSs are often analyzed by administrators who are tasked with countering the attack and minimizing its damage. Consequently, it is important that the alerts generated by the IDS are as detailed as possible. In this paper, we present a multi-layered behavior-based IDS using ensemble learning techniques for the classification of network attacks. Three widely adopted and appreciated models, i.e., Decision Trees, Random Forests, and Artificial Neural Networks, have been chosen to build the ensemble. To reduce the system response time, our solution is designed to immediately filter out trafic detected as benign without further analysis, while suspicious events are investigated to achieve a more fine-grained classification. Experimental evaluation performed on the CIC-IDS2017 public dataset shows that the system is able to detect nine categories of attacks with high performances, according to all the considered metrics.
not necessary for the detection of malicious trafic are discarded, while the actual detection of malicious trafic is performed in the third phase. Finally, the fourth phase concerns the mitigation measures taken after the detection of the attack. This last phase is not always performed by an automated system, but can be carried out directly by an administrator. Two diferent types of IDSs exist, depending on whether they analyze network trafic (network-based IDSs - NIDSs) or the behavior of individual hosts (host-based IDSs - HIDSs) by monitoring the state of the system through the analysis of the log files.
The authors of [
Statistical techniques and Machine learning can be used to overcome this limitation and
enable the design of systems that can automatically learn which events correspond to normal
or abnormal behaviors. In [
Machine and Deep Learning approaches, instead, allow the development of systems that
recognize patterns in malicious trafic and learn how to detect such trafic, as shown in [
Several IDSs exploit ensemble learning techniques for intrusion detection, as this allows for increased accuracy and classification performance over individual weak learners. For example, the authors of [19] propose the use of an ensemble learning technique in order to detect both known or new types of DDoS attacks. In this approach diferent base models focus on diferent aspects of intrusion. In [20], the authors use ensemble techniques to specifically detect botnet attacks against some protocols used in IoT networks. In particular, AdaBoost is used as an ensemble technique with three machine learning classifiers as weak learners: Decision Tree, Naive Bayes, and Artificial Neural Network. The system can detect only eight diferent types of botnet attacks. Many of the proposed systems only use a binary classification to identify anomalies. The lack of additional information about the specific type of attack often does not help administrators take appropriate countermeasures. In order to improve the decision making process, it may be useful to leverage a reputation system [21, 22] that takes into account the past behaviors of nodes in the network as proposed by [23].
Summarizing, multiclass IDSs in the literature generally recognize few attack classes, or are too specialized in the sense that they recognize only variants of the same attack. In contrast, our system achieves the right trade-of between number of recognized classes and prediction speed, as shown in the experimental results.
In this paper, we present a multi-layered Intrusion Detection System that uses machine learning and ensemble learning techniques to detect malicious trafic. The first layer provides a fast and eficient binary classification to recognize malicious trafic, while the second layer performs a more finely grained classification, identifying the attack as one of nine diferent types. In order to balance the dataset used and allow the system to correctly identify the classes represented by few samples, we propose the use of data augmentation techniques. The performances achieved by our system in terms of precision, accuracy, F1-score and FNR (false negative rate) are very high as shown by the extensive experimental evaluation.
The paper is organized as follows. Section 2 describes the proposed system and architecture. Section 3 explains details of the experiments and the results obtained from this study. Finally, Section 4 provides a conclusion of this work.
In this paper, we propose a novel multi-layered behavior-based IDS using ensemble learning that not only classifies trafic as benign or malicious, but also identifies trafic detected as malicious among nine possible categories of attacks.
The system architecture is composed of two layers and it is sketched in Figure 1. To prevent the system from being overloaded with all the network trafic, and consequently to prevent delayed detections, trafic filtering is preliminarily performed in order to distinguish “normal” Original features
Feature Selection 1 Feature Selection 2
Decision Tree
Normal traffic Abnormal traffic Random Forest
Decision Tree
Soft Voting Model
Output
Neural Network or “abnormal” trafic and pass to the second layer only the latter. For the design of the first layer, we decided to adopt a Decision Tree (DT), since experimental evaluation showed its better performance for binary classification, compared to other methods such as Neural Networks, Random Forest, and Gaussian Naive Bayes.
The choice of this classifier is particularly critical because it heavily influences the behavior of the whole IDS. Indeed, trafic classified as benign by the first layer is considered safe and is not further analyzed by subsequent layers. Thus, false negatives at this stage could have very detrimental efects on the entire network.
In the second layer, an accurate analysis of malicious trafic is performed so that the system can generate an alert as detailed as possible. A detailed alert [24] would provide the necessary information to administrators to allow them to neutralize the attack quickly and eficiently.
In particular, our solution uses ensemble learning techniques. This approach involves the use of diferent learning models, called weak learners. The results of the predictions of the single models are aggregated using appropriate ensemble techniques that yield better classification performances than those of the single weak learners. The weak learners we considered are Neural Networks (NNs), Random Forests (RFs) and Decision Trees (DTs). All the weak learners take as input the same features. The Random Forest is an ensemble method that uses Bagging as technique of aggregation and uses Decision Trees as weak learners. Bagging is a technique of homogeneous ensemble learning that is used to train diferent models and to aggregate the result using some kind of averaging.
The ensemble technique used to aggregate our weak learners is the so-called voting technique. Voting-based classifier is a type of heterogeneous ensemble learning. This type of ensemble works as an extension of Bagging. The architecture of a voting-based classifier is composed by homogeneous base models (in our case the Neural Network, the Random Forest and the Decision Tree) whose predictions are estimated in two diferent ways: hard and soft. In the hard voting mode, the logic of the majority is applied; consequently, the prediction considered correct by the ensemble will be the one that receives the highest number of votes from the weak learners. The soft voting mode considers, instead, the probability calculated from every base model that will be subsequently weighed. The prediction considered by the ensemble will be the one with the highest weighted probability.
One of the characteristics that makes the voting an optimal type of ensemble is its eficiency. Indeed, the full independence between the single base models allows the parallelization of the training phase of the weak learners, making both the training and prediction phases more eficient even in large scale scenarios. This feature is fundamental in systems that deal with anomaly detection, and in particular in intrusion detection systems, since even a slight delay in prediction could cause serious damage to devices connected to the network monitored by the IDS.
In this paper, we chose to use soft voting to merge the outputs of the individual classifiers, which are of diferent types. Specifically, the confidence values of the neural network prediction for each class is combined with the outputs of the Decision Tree and Random Forest. Future implementations may leverage more sophisticated approaches to assigning weight to classifiers, for example using reputation management systems [25, 26].
To extensively evaluate our system, the CIC-IDS2017 dataset was used. The details and principles behind its realization are described in [27]. Using the CICFlowMeter software, publicly available on the Canadian Institute for Cybersecurity website, more than 80 features were extracted and all trafic was labeled. The dataset is composed by eight trafic monitoring sessions organized into CSV files. These files contain records with features representing benign trafic and malicious trafic divided into diferent types of attacks. The trafic collected in the dataset contains 15 types of trafic: one represents normal trafic, while the others are diferent types of attacks.
In order to make the attack category computation less expensive, some classes have been grouped together. This makes the classification less detailed, but at the same time provides better performance in terms of time in detecting and identifying the attack.
The dataset contains several attacks belonging to the Web Attack class, such as SQL-Injection, Brute Force, XSS, and also several attacks belonging to the DoS class, such as DoS GoldenEye, DoS Hulk, DoS Slowhttptest, DoS slowloris. These attacks have been grouped, respectively, in Web Attack and DoS, in order to make the computation lighter while ensuring a rather detailed and precise identification of the malicious event.
Although this grouping causes a loss in the detail of attacks identification, it still allows the administrator to distinguish their category. It would be possible to obtain a more detailed alert on the attack subcategories by using an additional level of classification. However, this would
First Layer confusion Matrix L AM 99.97% R O N L A M R 0.04% O N B A 0.03% 99.96% NORMAL
ABNORMAL cause an increase in the computational cost of trafic detection and consequently an increase in system response time.
For the first classification layer, 80 relevant features of the dataset have been chosen, excluding the features containing data about source and destination IP addresses, timestamp, and flow ID. The latter contains a string that identifies the flow, consisting of the concatenation between the source and destination IP addresses, the source and destination ports, and the communication protocol. The exclusion of the above mentioned features allows the classifier to discern in the best way the malicious trafic from the benign one. In fact, considering source and destination IP addresses may lead the classifier to label a certain IP address as source or destination of malicious trafic, misclassifying further events, while the flow ID contains redundant information.
Several commonly adopted metrics have been used to evaluate our system, namely accuracy, precision, recall, F1-score; moreover, false negative rate was also considered because of the significant consequences an incorrect classification of malicious trafic could have. Various techniques for binary classification of trafic were implemented in several experiments. In order to perform a binary classification, all trafic in the dataset was relabeled using two labels: “Normal” and “Abnormal”. The best result for the first layer was obtained through the use of a Decision Tree.
In particular, after carrying out diferent implementations and having evaluated them using the main metrics, we choose to use a tree with maximum depth equal to 15. In fact, a further increase of the tree depth leads to an overfitting and consequently to a worsening of the main evaluation metrics. For the design of our IDS we have used as criterion for the evaluation of the quality of the split the technique of the information gain, that calculates the information acquired in relation to the feature used for the split of the current node of the Decision Tree.
The main results and the corresponding confusion matrix for the execution of the first layer of the system are reported in Table 1 and Figure 2, respectively.
For the training and testing phase of all models, we merged all eight trafic recording sessions, and then we randomly divided the resulting dataset into 70% for the training phase and 30% for the testing phase.
The nine classes considered are Bot, DDoS, DoS, FTP-patator, Heartbleed, Infiltration, Portscan, SSH-patator, Web Attack.
The analysis carried out by this layer takes into account a subset of the features considered in the previous layer. Among the various attack classes mentioned above, some are made up of a small number of samples. In particular, the Heartbleed and the Infiltration classes consist of 11 and 36 samples, respectively. Such a small number of samples, representing an insignificant amount compared to other classes containing hundreds of thousands of samples, does not allow the models to learn the patterns that distinguish them from other attacks.
Consequently, many samples belonging to such classes are not correctly classified by the models that constitute the second layer, thus causing a great loss of performance of the whole system.
In order to obtain a correct classification of such classes and a consequent increase of models performances, we used some techniques of data augmentation. These techniques augment the samples in the dataset by generating synthetic records from real data. In this work, we used the technique called ADASYN (Adaptive synthetic) sampling, presented in [28]. This technique implements a methodology that detects samples of minority class that are in a space dominated by the class with the largest number of elements.
In this way, synthetic samples are generated in areas of low density. This allows for more accurate classification of minority class elements, even in a region dominated by elements belonging to the most populated class.
After a careful evaluation and several experiments aimed to choose the best parameters, we the ensemble. Another classifier that performs well is the Decision Tree. Several experiments carried out by varying the parameters of the Decision Tree have shown good performance with a maximum depth of 50 and information gain as split criterion.
An increase in the maximum depth of the tree leads to an overfitting of the model and consequently to worse model performances. The performance achieved from this model is listed in Table 3 and Figure 4. The results show that the system achieves very high performance.
The lowest accuracy achieved is relative to the class of DoS attacks, which still maintains a percentage of 99.97%. The lowest values for precision and recall are related to the Web Attack class (more than 98%). This means that about 1% of the Web Attack trafic is mistaken by the classifier for DoS trafic.
We chose a Random Forest as the last classifier. The model used consists of 100 estimators, each with a maximum depth of 50, and information gain as split criterion. The results obtained by this model are shown in Table 4 and Figure 5. Even in this case, the performances are high. The lowest values for recall and precision are obtained for Web Attack class, but are still higher than 98%. Again, about 1% of the Web Attack trafic is mistaken by the classifier for a DoS attack. Finally, we present the results obtained by the entire ensemble after the soft voting phase.
The main metrics for the evaluation of the ensemble are shown in Figure 6 and Table 5. Since all the classifiers that make up the ensemble mistake a small percentage of the Web Attack trafic for a DoS attack, even in the ensemble about 1% of the Web Attack trafic is mistaken by the classifier for a DoS attack.
For the evaluation of the system, all the models that constitute the proposed IDS have been run 1000 times using diferent train and test sets at every execution. The results have been averaged and are shown in the tables and figures. All tests have been performed on of-the-shelf laptops equipped with Intel 3805U 1.9GHz CPU and 4GB RAM.
The execution times of the systems were estimated by averaging the prediction times of 10000 randomly chosen samples from the test set. The results of the execution times are shown in Table 6. The execution times of the entire system (both the first and second layers) have also been reported in this table. As can be seen, the total execution time is low, so the proposed system is suitable for real-time use.
In this work, we proposed a multi-layered behavior-based IDS to recognize malicious trafic and classify it into one of nine possible attack classes. The first of the two layers of the proposed system allows for preemptive trafic filtering through a very fast binary classification, which makes the whole IDS more eficient. Since malicious trafic generally constitutes a small percentage of the total trafic, this initial filtering greatly reduces the events analyzed by the second layer, improving the overall response time.
Numerous tests performed on the system have demonstrated its reliability and accuracy in detecting malicious trafic, as well as its time eficiency. Our IDS is able to recognize and identify 9 diferent types of attack in real-time, promptly alerting administrators to minimize serious consequences. Future implementations may provide an additional layer of classification that more finely identifies the type of DoS or Web attacks. However, this would require a further computational cost and, consequently, a delay in the identification of the attack. It is always necessary to make a trade-of between the granularity of the classification and the computational cost. A possible solution could be to immediately send an alert if one of the grouped attack categories is recognized, and then subsequently perform a more detailed analysis, performing a two-stage classification.
This work is partially funded by the European Union - FESR o FSE, PON Ricerca e Innovazione 2014-2020 - DM 1062/2021. IEEE Access 9 (2021) 101574–101599. doi:10.1109/ACCESS.2021.3097247. [16] V. Agate, F. Concone, P. Ferraro, A resilient smart architecture for road surface condition monitoring, in: The Proceedings of the International Conference on Smart City Applications, Springer, 2021, pp. 199–209. [17] G. Mylavarapu, J. Thomas, A. K. TK, Real-time hybrid intrusion detection system using apache storm, in: 2015 IEEE 7th International Symposium on Cyberspace Safety and Security, 2015, pp. 1436–1441. doi:10.1109/HPCC-CSS-ICESS.2015.241. [18] C. Ma, X. Du, L. Cao, Analysis of multi-types of flow features based on hybrid neural network for improving network anomaly detection, IEEE Access 7 (2019) 148363–148380. doi:10.1109/ACCESS.2019.2946708. [19] S. Das, A. M. Mahfouz, D. Venugopal, S. Shiva, Ddos intrusion detection through machine learning ensemble, in: 2019 IEEE 19th International Conference on Software Quality, Reliability and Security Companion (QRS-C), 2019, pp. 471–477. doi:10.1109/QRS-C. 2019.00090. [20] N. Moustafa, B. Turnbull, K.-K. R. Choo, An ensemble intrusion detection technique based on proposed statistical flow features for protecting network trafic of internet of things, IEEE Internet of Things Journal 6 (2019) 4815–4830. doi:10.1109/JIOT.2018.2871719. [21] V. Agate, A. De Paola, G. Lo Re, M. Morana, Vulnerability Evaluation of Distributed Reputation Management Systems, in: InfQ 2016 - New Frontiers in Quantitative Methods in Informatics, ICST (Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering), ICST, Brussels, Belgium, Belgium, 2016, pp. 1–8. [22] V. Agate, A. De Paola, S. Gaglio, G. Lo Re, M. Morana, A framework for parallel assessment of reputation management systems, in: Proceedings of the 17th International Conference on Computer Systems and Technologies 2016, 2016, pp. 121–128. [23] K. Gerrigagoitia, R. Uribeetxeberria, U. Zurutuza, I. Arenaza, Reputation-based intrusion detection system for wireless sensor networks, in: 2012 Complexity in Engineering (COMPENG). Proceedings, 2012, pp. 1–5. doi:10.1109/CompEng.2012.6242969. [24] A. De Paola, P. Ferraro, S. Gaglio, G. Lo Re, M. Morana, M. Ortolani, D. Peri, A contextaware system for ambient assisted living, in: International Conference on Ubiquitous Computing and Ambient Intelligence, Springer, 2017, pp. 426–438. [25] V. Agate, A. D. Paola, G. Lo Re, M. Morana, A simulation software for the evaluation of vulnerabilities in reputation management systems, ACM Transactions on Computer Systems (TOCS) 37 (2021) 1–30. [26] V. Agate, A. De Paola, G. L. Re, M. Morana, A platform for the evaluation of distributed reputation algorithms, in: 2018 IEEE/ACM 22nd International Symposium on Distributed Simulation and Real Time Applications (DS-RT), IEEE, 2018, pp. 1–8. [27] I. Sharafaldin, A. H. Lashkari, A. A. Ghorbani, Toward generating a new intrusion detection dataset and intrusion trafic characterization. (2018). [28] H. He, Y. Bai, E. A. Garcia, S. Li, Adasyn: Adaptive synthetic sampling approach for imbalanced learning, in: 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence), 2008, pp. 1322–1328. doi:10.1109/ IJCNN.2008.4633969.