Adaptive Ensemble Learning for Intrusion Detection Systems Vincenzo Agate, Federico Concone* , Alessandra De Paola, Pierluca Ferraro, Salvatore Gaglio, Giuseppe Lo Re and Marco Morana Università degli Studi di Palermo, Dipartimento di Ingegneria, Palermo, Italy Abstract For years, the European Commission has highlighted the need to invest in cybersecurity as a means of protecting institutions and citizens from the many threats in cyberspace. Attacks perpetrated through the network are extremely dangerous, also because their mitigation is complex, making it difficult to ensure an adequate level of security. One of the crucial elements in building an overall system of protection against network-based cyber attacks are Intrusion Detection Systems (IDSs), whose goal is to detect and identify such attacks and misuse of computer networks in a timely manner. Nowadays, the most effective IDSs are based on Machine Learning (ML) and are able to combine and analyze information from heterogeneous sources, such as network traffic, user activity patterns, and data extracted from system logs. However, these tools commonly exploit specific classifiers, whose performance is highly dependent on the attacks being considered, and are unable to generalize adequately enough to be applied in different contexts. The research laboratories of Networking and Distributed Systems and Artificial Intelligence at the University of Palermo are carrying out research activities in order to address these issues, with the main goal of designing a new generation of IDSs that, by dynamically and adaptively combining multiple classifiers, are able to overcome the limitations of state-of-the-art solutions. Keywords Cybersecurity, Artificial Intelligence, Intrusion Detection Systems 1. Introduction will face in the near future is the adoption of Machine Learning (ML) and, more generally, Artificial Intelligence Today, with the increasingly pervasive use of ICT tech- (AI) methods. nologies, cyber attacks pose a serious risk to the infras- However, a thorough study of the literature shows that tructural, productive and economic aspects of our soci- the adoption of machine learning methods to design IDSs ety. One of the most critical threats to today’s hyper- involves several critical issues. One of the most notice- connected world are attacks that come from the network. able concerns is that, due to the high heterogeneity of In fact, all social and productive realities are closely de- network traffic generated by different attacks, specific pendent on the ability to exchange data through the net- classifiers are characterized by performance that is highly work. This dependence can be exploited by the malicious dependent on the attacks considered. This means that parties to gain unauthorized access to the resources of there is no single universal ML approach that can detect institutions and organizations. One of the most effective any kind of attack in different scenarios. In addition, solutions to such attacks are Intrusion Detection Systems different classes of ML approaches have very different ca- (IDSs), whose main goal is to timely detect and iden- pabilities: for example, supervised methods can achieve tify misuse of resources early enough to enable timely excellent performance but are unable to handle unknown responses that stop any malicious behavior and ensure attacks, while unsupervised methods can detect anoma- normal operation of systems. lies and unknown attacks but generally achieve poor Currently, the most promising approach to designing performance with already known intrusions [1]. IDSs capable of dealing with the threats our systems The adoption of ensemble machine learning tech- niques, which leverage multiple machine learning al- Ital-IA 2024: 4th National Conference on Artificial Intelligence, orga- nized by CINI, May 29-30, 2024, Naples, Italy gorithms, promises to be a very effective approach to * Corresponding author. achieve higher overall performance than single methods. $ vincenzo.agate@unipa.it (V. Agate); federico.concone@unipa.it However, in the current literature, the ensemble of classi- (F. Concone); alessandra.depaola@unipa.it (A. De Paola); fiers is often designed through trial-and-error procedures, pierluca.ferraro@unipa.it (P. Ferraro); salvatore.gaglio@unipa.it and there is no evidence that an approach suitable for a (S. Gaglio); giuseppe.lore@unipa.it (G. Lo Re); marco.morana@unipa.it (M. Morana) specific scenario can be general enough to be adopted in  0000-0002-3326-8500 (V. Agate); 0000-0001-7638-3624 different scenarios. (F. Concone); 0000-0002-7340-1847 (A. De Paola); Our research group, through scientific activities 0000-0003-1574-1111 (P. Ferraro); 0000-0002-8217-2230 (G. Lo Re); funded by various projects, seeks to contribute to this 0000-0002-5963-6236 (M. Morana) research area by designing new methodologies and adap- © 2024 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). CEUR ceur-ws.org Workshop ISSN 1613-0073 Proceedings tive solutions aiming to improve the robustness of exist- selection method and a ranking technique that evaluates ing approaches in the field of AI- and ML-based intrusion the ability of different base classifiers to detect different detection systems (IDS). attacks. Results are promising, but only for a subset of The following of this paper introduces the current state the considered attack classes. The authors of [9] propose of the art of IDS and discusses the main limitations of a model based on sustainable ensemble learning and on current solutions, followed by a summary description of incremental learning. Such a system exploits multiclass our research group’s contribution. Finally, a description regression models so that the ensemble is adapted to rec- of the challenges and goals we intend to address in the ognize different types of attacks; moreover, by means near future is provided. of an iterative update method the parameters and the decision results of the historical model are included into the training process of the final ensemble model. 2. Related Work The performances of the solutions described above, as well as many other existing ensemble frameworks, are In the dynamic domain of cybersecurity, the arms race be- severely limited as many different classes of attacks can tween intrusion detection mechanisms and cyber-attack occur. Moreover, the combination of multiple ML-based methodologies has accelerated, highlighting an urgent classifiers generally increases the computational load, need for innovative detection techniques. Several IDSs thus limiting the IDS’s ability to operate timely. This have been proposed in the literature, exploiting both issue is particularly critical, given the need to promptly signature-based and anomaly-based approaches [2, 3]. identify incoming threats and immediately apply appro- The former are reliable in recognizing known attacks but priate countermeasures. are ineffective against those not previously seen. Con- versely, the latter show a more flexible behavior and are better suited to detect constantly evolving attacks, espe- 3. Research Contribution cially by using Machine Learning (ML) techniques. Nevertheless, the design of ML-based IDSs faces sev- In this perspective, a first contribution of our research eral challenges, such as the difficulty of ensuring fast unit is discussed in [10], where we introduced a system responses when dealing with high-dimensional data, as which addresses critical limitations in existing frame- in the case of network traffic, or providing consistently works, achieving the right trade-off between number of good performance for all types of intrusions. Moreover, recognized classes and prediction speed, in contrast to in modern network environments with heterogeneous other multi-class IDSs in the literature. devices, the input data distributions are subject to un- In particular, we presented a multi-layered architecture predictable fluctuations over time. This phenomenon, for a behavior-based Intrusion Detection System that referred to as concept drift, poses a significant challenge uses machine learning and ensemble learning techniques in the fields of machine learning and cybersecurity, as to distinguish between benign and malicious traffic and noted in [4]. One of the most promising directions to categorize detected malicious activities into one of nine achieve overall good performance is the adoption of en- possible attack classes. The architecture of the system is semble learning techniques [5], which exploit multiple shown in Figure 1. ML algorithms to obtain better results than those of indi- The experimental evaluation was performed on the vidual methods. CIC-IDS2017 public dataset, showing that the proposed The IDS presented in [6], for instance, combines a two- IDS exhibits good performance in detecting all attack stage meta classifier ensemble (i.e., rotation forest and classes according to well-established metrics. bagging) with hybrid feature selection (particle swarm A key aspect of our proposed system is its two-layer ar- optimization, ant colony algorithm, and genetic algo- chitecture. To prevent the system from being overloaded rithm) to better distinguish regular and anomalous traf- with all the network traffic, and consequently to prevent fic. However, such a solution is tailored on single attacks delayed detections, traffic filtering is preliminarily per- instances and not suitable for dealing with multi-class formed in order to distinguish “normal” and “abnormal” problems. The IDS introduced in [7] adopts an ensemble traffic, ensuring that only potentially malicious traffic approach that combines decision trees, Random Forest, is advanced to the next stage for further analysis. This and Forest by Penalizing Attributes algorithms, and a layer thus acts as a filter, improving the efficiency of the voting technique to combine their probability distribu- whole system. Accurate classification at this stage is cru- tions. Although the system achieves good performance cial, as traffic deemed benign is not subject to subsequent with popular attacks, this drops in the case of rare ones. scrutiny, highlighting the importance of minimizing false Multi-class intrusion detection is also addressed in [8], negatives to safeguard network integrity. For the design where an ensemble approach is designed to detect dif- of the first layer, we decided to adopt a Decision Tree ferent attacks. Such IDS also exploits a hybrid feature (DT), since experimental evaluation showed its better per- Original features Feature Selection 1 First Layer Decision Tree Normal traffic Abnormal traffic Feature Selection 2 Random Forest Second Layer Decision Tree Soft Voting Model Output Neural Network Figure 1: Architecture of the multi-layered IDS proposed in [10]. formance for binary classification, compared to Neural lelization in the training and testing of weak learners, Networks, Random Forest, and Gaussian Naive Bayes. thereby enhancing efficiency in both training and pre- In the second layer, a detailed analysis of malicious diction phases, a critical feature for IDS systems where traffic is performed so thus the system generates alerts timely threat detection is paramount. more accurately. These alerts provide network admin- This work is partially funded by the European Union - istrators with the information they need to quickly and FESR o FSE, PON Ricerca e Innovazione 2014-2020 - DM effectively respond to threats [11], allowing them to neu- 1062/2021. tralize ongoing attacks quickly and efficiently. Our solution proposes the adoption of ensemble learn- ing techniques, incorporating a combination of different 4. Preliminary Evaluation learning models, such as Neural Networks (NNs), Ran- To conduct a preliminary evaluation of the proposed solu- dom Forests (RFs), and additional DTs as weak learners. tion, the CIC-IDS2017 dataset was used [12]. This dataset The results of the predictions of the single models are perfectly fits the goals of our study as it includes var- aggregated using appropriate ensemble techniques that ious attacks encompassing SQL-Injection, Brute Force, yield better classification performances than those of the XSS, DoS GoldenEye, DoS Hulk, DoS Slowhttptest, and single weak learners. Specifically, we adopt a weighted DoS Slowloris. These attacks were grouped under two voting technique that assigns higher weights to the pre- categories, i.e., Web and DOS Attacks, to streamline com- dictions of classifiers with low uncertainty in order to putation while maintaining detailed and accurate identi- determine the ensemble’s final verdict. fication of malicious events. The adoption of this weighted voting strategy for ag- All tests have been performed on off-the-shelf laptops gregating classifier outputs, integrating the confidence equipped with Intel 3805U 1.9GHz CPU and 4GB RAM. values from neural network predictions with those of Moreover, all the models that constitute the proposed Decision Trees and Random Forests, notably improves IDS have been run 1000 times using different train and the performance of the whole IDS. Finally, it is worth test sets at every execution. noticing that our system’s architecture facilitates paral- The numerous tests performed on the system have chine learning models. Indeed, ignoring the phenomenon demonstrated its reliability and accuracy in detecting of concept drift, like many current IDSs do, inevitably malicious traffic, as well as its time efficiency. The IDS lead to performance degradation over time. is able to recognize and identify 9 different types of at- Our future approach will try to overcome these chal- tack in real-time, promptly alerting administrators to lenges by orchestrating supervised and unsupervised minimize serious consequences. In fact, on average, the systems to exploit the benefits of both approaches. The system misses attacks in very small percentages (close to detection of unknown attacks can rely on online unsu- 1%), while it requires extremely low execution time for pervised anomaly detection systems that are adept at both the first and second levels: some slight difference recognizing signs of zero-day attacks, all the while auto- is appreciated in dependence on the model used in the matically adapting to concept drift without the constant ensemble. need for manual intervention. This, in turn, can also Besides the good performance achieved, numerous reduce the frequency of model re-training and enhance improvements are needed to address other important system efficiency. Such systems will be used in conjunc- limitations, that are common to many IDSs in the litera- tion with supervised ones to improve the overall accuracy ture. for known attacks. First of all, the solutions proposed in the literature (as The efficacy of our methodologies will be validated well as [10]) select the set of classifiers to be adopted through extensive experimental evaluation, showcasing through a trial-and-error process and lack a formalized our system’s capability of real-time threat detection com- methodology that can drive the design process in differ- pared to traditional models. This will provide the re- ent scenarios. Moreover, many of the existing solutions search community with valuable insights into the effec- have been designed ignoring the outbreak of unknown tiveness of different ML methods and ensemble strategies attacks. Such a “closed-world” approach makes IDSs un- against a wide range of security attacks. suitable for recognizing special types of attacks known Looking forward, we envision further enriching our as “zero-day”. IDS framework to improve its resilience against unknown attacks and concept drift, offering robust defenses against the ever-evolving landscape of cyber threats. 5. Challenges and Goals The main goal of the research unit is the design and 6. Research Unit development of a novel class of IDSs based on the com- bination of several dynamically orchestrated classifiers The Networks and Distributed Systems and Artificial Intel- (both supervised and unsupervised), with the aim of rec- ligence research laboratories at the University of Palermo, ognizing a large set of different threats, also detecting directed by Prof. Giuseppe Lo Re and Salvatore Gaglio, the occurrence of zero-day attacks. have experience in several research fields such as dis- Given the strong characterization of the many appli- tributed systems, cybersecurity, artificial intelligence, cation scenarios in which IDSs are needed, the design and machine learning. In particular, the research unit of the system architecture will be guided by a formal- has developed deep expertise in several topics related to ized, rigorous, and replicable approach that can steer the cybersecurity domain that mainly concern the adop- the realization of specific IDS instances. The goal is to tion of artificial intelligence to assist the detection and design a scalable and modular architecture, capable of identification of potential threats in cyberspace. The iden- maintaining a low computing load while guaranteeing tified methodologies and proposed solutions have been high detection performance and responsiveness, even in applied in different scenarios, such as intrusion detection the presence of huge amounts of data. systems [10], malware detection systems [13, 14], social The main challenge will be the definition of adaptive network security [15, 16], privacy-preserving distributed orchestration techniques, which will be crucial for the systems [17, 18], adversarial machine learning [19] and design of IDSs capable of dynamically adjusting their secure crowdsensing [20]. ensemble strategies based on the observed context. This Furthermore, it is worth noting that the research will include the integration of both supervised and un- group’s experience in applying artificial intelligence ap- supervised learning approaches, allowing an adaptive proaches and methods to distributed systems and cy- response to emerging threats. bersecurity challenges has been leveraged in several To reach this ambitious goal, the system will also have funded research projects, such as FRASI - FRamework to address the phenomenon of concept drift, which is the for Agent-based Semantic- aware In-teroperability (FAR continuous shift of the statistical distribution of network MIUR D.M. 8 agosto 2000), Bigger Data (D.D. MIUR n. data over time. This poses a big challenge for current 2690 dell’11.12.2013, Piano di Azione e Coesione), SeN- IDSs, often necessitating manual retraining of their ma- Sori - SEnsor Node as a Service for hOme and buildings eneRgy savIng (Industria 2015: Bando Nuove Tecnolo- [8] S. Seth, K. K. Chahal, G. Singh, A novel ensem- gie per il Made in Italy), Smart Buildings - An Ambient ble framework for an intelligent intrusion detec- Intelligence system for optimizing energy resources in tion system, IEEE Access 9 (2021) 138451–138467. building complexes (PO FESR Sicilia 2007-2013), OnSi- doi:10.1109/ACCESS.2021.3116219. cily.com - a Web 3.0 platform with intelligent virtual [9] X. Li, M. Zhu, L. T. Yang, M. Xu, Z. Ma, C. Zhong, A.V.I. assistance (PO FESR Sicilia 2007-2013), VASARI - H. Li, Y. Xiang, Sustainable ensemble learning driv- VAlorizzazione Smart del patrimonio ARtistico delle città ing intrusion detection model, IEEE Transactions Italiane (PNR 2015-2020), CrowdSense (PO FESR Sicilia on Dependable and Secure Computing 18 (2021) 2014-2020), Smart Wave (PO FESR Sicilia 2014-2020), S6 1591–1604. doi:10.1109/TDSC.2021.3066202. Project - A Smart, Social and SDN-based Surveillance [10] V. Agate, D. Felice Maria, A. De Paola, P. Ferraro, System for Smart-cities (PO FESR Sicilia 2014-2020), S3 G. Lo Re, M. Morana, A behavior-based intrusion de- Campus - SHARING, SMART AND SUSTAINABLE CAM- tection system using ensemble learning techniques., PUS (POC Sicilia 2014-2020 ), Smart Venues for Agrotech in: ITASEC, 2022, pp. 207–218. Ecosystem (POC Sicilia 2014-2020). [11] A. De Paola, P. Ferraro, S. Gaglio, G. Lo Re, M. Morana, M. Ortolani, D. Peri, A context-aware system for ambient assisted living, in: S. F. Ochoa, References P. Singh, J. Bravo (Eds.), Ubiquitous Computing and Ambient Intelligence, Springer International [1] T. Zoppi, A. Ceccarelli, T. Puccetti, A. Bondavalli, Publishing, Cham, 2017, pp. 426–438. Which algorithm can detect unknown attacks? [12] I. Sharafaldin, A. H. Lashkari, A. A. Ghorbani, To- comparison of supervised, unsupervised and meta- ward generating a new intrusion detection dataset learning algorithms for intrusion detection, Com- and intrusion traffic characterization, in: Proceed- puters & Security 127 (2023) 103107. ings of the 4th International Conference on Infor- [2] A. Khraisat, I. Gondal, P. Vamplew, J. Kamruzzaman, mation Systems Security and Privacy - Volume 1: Survey of intrusion detection systems: techniques, ICISSP„ INSTICC, SciTePress, 2018, pp. 108–116. datasets and challenges, Cybersecurity 2 (2019) doi:10.5220/0006639801080116. 20. URL: https://doi.org/10.1186/s42400-019-0038-7. [13] A. De Paola, S. Gaglio, G. Lo Re, M. Morana, A hy- doi:10.1186/s42400-019-0038-7. brid system for malware detection on big data, in: [3] A. L. Buczak, E. Guven, A survey of data min- IEEE INFOCOM 2018 - IEEE Conference on Com- ing and machine learning methods for cyber se- puter Communications Workshops (INFOCOM WK- curity intrusion detection, IEEE Communications SHPS), 2018, pp. 45–50. doi:10.1109/INFCOMW. Surveys & Tutorials 18 (2016) 1153–1176. doi:10. 2018.8406963. 1109/COMST.2015.2494502. [14] F. Concone, A. De Paola, G. Lo Re, M. Morana, Twit- [4] J. Lu, A. Liu, F. Dong, F. Gu, J. Gama, G. Zhang, ter analysis for real-time malware discovery, in: Learning under concept drift: A review, IEEE Trans- 2017 AEIT International Annual Conference (2017 actions on Knowledge and Data Engineering 31 AEIT), Cagliari, Italy, 2017. (2018) 2346–2363. [15] F. Concone, G. Lo Re, M. Morana, S. K. Das, Spade: [5] A. A. Aburomman, M. B. I. Reaz, A survey of intru- Multi-stage spam account detection for online so- sion detection systems based on ensemble and hy- cial networks, IEEE Transactions on Dependable brid classifiers, Computers & Security 65 (2017) 135– and Secure Computing (2022) 1–16. doi:10.1109/ 152. URL: https://www.sciencedirect.com/science/ TDSC.2022.3198830. article/pii/S0167404816301572. doi:https://doi. [16] F. Concone, G. Lo Re, M. Morana, C. Ruocco, Twitter org/10.1016/j.cose.2016.11.004. spam account detection by effective labeling, in: [6] B. A. Tama, M. Comuzzi, K.-H. Rhee, Tse-ids: A two- 3rd Italian Conference on Cyber Security, ITASEC stage classifier ensemble for intelligent anomaly- 2019, volume 2315, IT, 2019. based intrusion detection system, IEEE Access 7 [17] V. Agate, P. Ferraro, G. Lo Re, S. K. Das, (2019) 94497–94507. doi:10.1109/ACCESS.2019. Blind: A privacy preserving truth discovery 2928048. system for mobile crowdsensing, Journal of [7] Y. Zhou, G. Cheng, S. Jiang, M. Dai, Building Network and Computer Applications (2023) an efficient intrusion detection system based 103811. URL: https://www.sciencedirect.com/ on feature selection and ensemble classifier, science/article/pii/S1084804523002308. doi:https: Computer Networks 174 (2020) 107247. URL: //doi.org/10.1016/j.jnca.2023.103811. https://www.sciencedirect.com/science/article/pii/ [18] V. Agate, A. De Paola, P. Ferraro, G. Lo Re, S1389128619314203. doi:https://doi.org/10. M. Morana, Secureballot: A secure open 1016/j.comnet.2020.107247. source e-voting system, Journal of Network and Computer Applications 191 (2021) 103165. URL: https://www.sciencedirect.com/science/ article/pii/S1084804521001776. doi:https: //doi.org/10.1016/j.jnca.2021.103165. [19] S. Gaglio, A. Giammanco, G. Lo Re, M. Morana, Adversarial machine learning in e-health: attack- ing a smart prescription system, in: International Conference of the Italian Association for Artificial Intelligence (2021 AI*IA), Milan, Italy, 2021. [20] F. Concone, G. Lo Re, M. Morana, Smcp: a secure mobile crowdsensing protocol for fog- based applications, Human-centric Computing and Information Sciences 10 (2020) 1–23. URL: https://doi.org/10.1186/s13673-020-00232-y. doi:10. 1186/s13673-020-00232-y.