=Paper=
{{Paper
|id=Vol-3762/475
|storemode=property
|title=Adaptive Ensemble Learning for Intrusion Detection Systems
|pdfUrl=https://ceur-ws.org/Vol-3762/475.pdf
|volume=Vol-3762
|authors=Vincenzo Agate,Federico Concone,Alessandra De Paola,Pierluca Ferraro,Salvatore Gaglio,Giuseppe Lo Re,Marco Morana
|dblpUrl=https://dblp.org/rec/conf/ital-ia/AgateCPFGRM24
}}
==Adaptive Ensemble Learning for Intrusion Detection Systems==
https://ceur-ws.org/Vol-3762/475.pdf
Adaptive Ensemble Learning for Intrusion Detection
Systems
Vincenzo Agate, Federico Concone* , Alessandra De Paola, Pierluca Ferraro, Salvatore Gaglio,
Giuseppe Lo Re and Marco Morana
Università degli Studi di Palermo, Dipartimento di Ingegneria, Palermo, Italy
Abstract
For years, the European Commission has highlighted the need to invest in cybersecurity as a means of protecting institutions
and citizens from the many threats in cyberspace. Attacks perpetrated through the network are extremely dangerous, also
because their mitigation is complex, making it difficult to ensure an adequate level of security. One of the crucial elements in
building an overall system of protection against network-based cyber attacks are Intrusion Detection Systems (IDSs), whose
goal is to detect and identify such attacks and misuse of computer networks in a timely manner. Nowadays, the most effective
IDSs are based on Machine Learning (ML) and are able to combine and analyze information from heterogeneous sources,
such as network traffic, user activity patterns, and data extracted from system logs. However, these tools commonly exploit
specific classifiers, whose performance is highly dependent on the attacks being considered, and are unable to generalize
adequately enough to be applied in different contexts. The research laboratories of Networking and Distributed Systems and
Artificial Intelligence at the University of Palermo are carrying out research activities in order to address these issues, with the
main goal of designing a new generation of IDSs that, by dynamically and adaptively combining multiple classifiers, are able
to overcome the limitations of state-of-the-art solutions.
Keywords
Cybersecurity, Artificial Intelligence, Intrusion Detection Systems
1. Introduction will face in the near future is the adoption of Machine
Learning (ML) and, more generally, Artificial Intelligence
Today, with the increasingly pervasive use of ICT tech- (AI) methods.
nologies, cyber attacks pose a serious risk to the infras- However, a thorough study of the literature shows that
tructural, productive and economic aspects of our soci- the adoption of machine learning methods to design IDSs
ety. One of the most critical threats to today’s hyper- involves several critical issues. One of the most notice-
connected world are attacks that come from the network. able concerns is that, due to the high heterogeneity of
In fact, all social and productive realities are closely de- network traffic generated by different attacks, specific
pendent on the ability to exchange data through the net- classifiers are characterized by performance that is highly
work. This dependence can be exploited by the malicious dependent on the attacks considered. This means that
parties to gain unauthorized access to the resources of there is no single universal ML approach that can detect
institutions and organizations. One of the most effective any kind of attack in different scenarios. In addition,
solutions to such attacks are Intrusion Detection Systems different classes of ML approaches have very different ca-
(IDSs), whose main goal is to timely detect and iden- pabilities: for example, supervised methods can achieve
tify misuse of resources early enough to enable timely excellent performance but are unable to handle unknown
responses that stop any malicious behavior and ensure attacks, while unsupervised methods can detect anoma-
normal operation of systems. lies and unknown attacks but generally achieve poor
Currently, the most promising approach to designing performance with already known intrusions [1].
IDSs capable of dealing with the threats our systems The adoption of ensemble machine learning tech-
niques, which leverage multiple machine learning al-
Ital-IA 2024: 4th National Conference on Artificial Intelligence, orga-
nized by CINI, May 29-30, 2024, Naples, Italy
gorithms, promises to be a very effective approach to
*
Corresponding author. achieve higher overall performance than single methods.
$ vincenzo.agate@unipa.it (V. Agate); federico.concone@unipa.it However, in the current literature, the ensemble of classi-
(F. Concone); alessandra.depaola@unipa.it (A. De Paola); fiers is often designed through trial-and-error procedures,
pierluca.ferraro@unipa.it (P. Ferraro); salvatore.gaglio@unipa.it and there is no evidence that an approach suitable for a
(S. Gaglio); giuseppe.lore@unipa.it (G. Lo Re);
marco.morana@unipa.it (M. Morana)
specific scenario can be general enough to be adopted in
0000-0002-3326-8500 (V. Agate); 0000-0001-7638-3624 different scenarios.
(F. Concone); 0000-0002-7340-1847 (A. De Paola); Our research group, through scientific activities
0000-0003-1574-1111 (P. Ferraro); 0000-0002-8217-2230 (G. Lo Re); funded by various projects, seeks to contribute to this
0000-0002-5963-6236 (M. Morana) research area by designing new methodologies and adap-
© 2024 Copyright for this paper by its authors. Use permitted under Creative Commons License
Attribution 4.0 International (CC BY 4.0).
CEUR
ceur-ws.org
Workshop ISSN 1613-0073
Proceedings
tive solutions aiming to improve the robustness of exist- selection method and a ranking technique that evaluates
ing approaches in the field of AI- and ML-based intrusion the ability of different base classifiers to detect different
detection systems (IDS). attacks. Results are promising, but only for a subset of
The following of this paper introduces the current state the considered attack classes. The authors of [9] propose
of the art of IDS and discusses the main limitations of a model based on sustainable ensemble learning and on
current solutions, followed by a summary description of incremental learning. Such a system exploits multiclass
our research group’s contribution. Finally, a description regression models so that the ensemble is adapted to rec-
of the challenges and goals we intend to address in the ognize different types of attacks; moreover, by means
near future is provided. of an iterative update method the parameters and the
decision results of the historical model are included into
the training process of the final ensemble model.
2. Related Work The performances of the solutions described above, as
well as many other existing ensemble frameworks, are
In the dynamic domain of cybersecurity, the arms race be-
severely limited as many different classes of attacks can
tween intrusion detection mechanisms and cyber-attack
occur. Moreover, the combination of multiple ML-based
methodologies has accelerated, highlighting an urgent
classifiers generally increases the computational load,
need for innovative detection techniques. Several IDSs
thus limiting the IDS’s ability to operate timely. This
have been proposed in the literature, exploiting both
issue is particularly critical, given the need to promptly
signature-based and anomaly-based approaches [2, 3].
identify incoming threats and immediately apply appro-
The former are reliable in recognizing known attacks but
priate countermeasures.
are ineffective against those not previously seen. Con-
versely, the latter show a more flexible behavior and are
better suited to detect constantly evolving attacks, espe- 3. Research Contribution
cially by using Machine Learning (ML) techniques.
Nevertheless, the design of ML-based IDSs faces sev- In this perspective, a first contribution of our research
eral challenges, such as the difficulty of ensuring fast unit is discussed in [10], where we introduced a system
responses when dealing with high-dimensional data, as which addresses critical limitations in existing frame-
in the case of network traffic, or providing consistently works, achieving the right trade-off between number of
good performance for all types of intrusions. Moreover, recognized classes and prediction speed, in contrast to
in modern network environments with heterogeneous other multi-class IDSs in the literature.
devices, the input data distributions are subject to un- In particular, we presented a multi-layered architecture
predictable fluctuations over time. This phenomenon, for a behavior-based Intrusion Detection System that
referred to as concept drift, poses a significant challenge uses machine learning and ensemble learning techniques
in the fields of machine learning and cybersecurity, as to distinguish between benign and malicious traffic and
noted in [4]. One of the most promising directions to categorize detected malicious activities into one of nine
achieve overall good performance is the adoption of en- possible attack classes. The architecture of the system is
semble learning techniques [5], which exploit multiple shown in Figure 1.
ML algorithms to obtain better results than those of indi- The experimental evaluation was performed on the
vidual methods. CIC-IDS2017 public dataset, showing that the proposed
The IDS presented in [6], for instance, combines a two- IDS exhibits good performance in detecting all attack
stage meta classifier ensemble (i.e., rotation forest and classes according to well-established metrics.
bagging) with hybrid feature selection (particle swarm A key aspect of our proposed system is its two-layer ar-
optimization, ant colony algorithm, and genetic algo- chitecture. To prevent the system from being overloaded
rithm) to better distinguish regular and anomalous traf- with all the network traffic, and consequently to prevent
fic. However, such a solution is tailored on single attacks delayed detections, traffic filtering is preliminarily per-
instances and not suitable for dealing with multi-class formed in order to distinguish “normal” and “abnormal”
problems. The IDS introduced in [7] adopts an ensemble traffic, ensuring that only potentially malicious traffic
approach that combines decision trees, Random Forest, is advanced to the next stage for further analysis. This
and Forest by Penalizing Attributes algorithms, and a layer thus acts as a filter, improving the efficiency of the
voting technique to combine their probability distribu- whole system. Accurate classification at this stage is cru-
tions. Although the system achieves good performance cial, as traffic deemed benign is not subject to subsequent
with popular attacks, this drops in the case of rare ones. scrutiny, highlighting the importance of minimizing false
Multi-class intrusion detection is also addressed in [8], negatives to safeguard network integrity. For the design
where an ensemble approach is designed to detect dif- of the first layer, we decided to adopt a Decision Tree
ferent attacks. Such IDS also exploits a hybrid feature (DT), since experimental evaluation showed its better per-
Original features
Feature
Selection 1 First Layer
Decision Tree
Normal traffic
Abnormal traffic
Feature
Selection 2
Random Forest Second Layer
Decision Tree Soft Voting Model
Output
Neural Network
Figure 1: Architecture of the multi-layered IDS proposed in [10].
formance for binary classification, compared to Neural lelization in the training and testing of weak learners,
Networks, Random Forest, and Gaussian Naive Bayes. thereby enhancing efficiency in both training and pre-
In the second layer, a detailed analysis of malicious diction phases, a critical feature for IDS systems where
traffic is performed so thus the system generates alerts timely threat detection is paramount.
more accurately. These alerts provide network admin- This work is partially funded by the European Union -
istrators with the information they need to quickly and FESR o FSE, PON Ricerca e Innovazione 2014-2020 - DM
effectively respond to threats [11], allowing them to neu- 1062/2021.
tralize ongoing attacks quickly and efficiently.
Our solution proposes the adoption of ensemble learn-
ing techniques, incorporating a combination of different 4. Preliminary Evaluation
learning models, such as Neural Networks (NNs), Ran-
To conduct a preliminary evaluation of the proposed solu-
dom Forests (RFs), and additional DTs as weak learners.
tion, the CIC-IDS2017 dataset was used [12]. This dataset
The results of the predictions of the single models are
perfectly fits the goals of our study as it includes var-
aggregated using appropriate ensemble techniques that
ious attacks encompassing SQL-Injection, Brute Force,
yield better classification performances than those of the
XSS, DoS GoldenEye, DoS Hulk, DoS Slowhttptest, and
single weak learners. Specifically, we adopt a weighted
DoS Slowloris. These attacks were grouped under two
voting technique that assigns higher weights to the pre-
categories, i.e., Web and DOS Attacks, to streamline com-
dictions of classifiers with low uncertainty in order to
putation while maintaining detailed and accurate identi-
determine the ensemble’s final verdict.
fication of malicious events.
The adoption of this weighted voting strategy for ag-
All tests have been performed on off-the-shelf laptops
gregating classifier outputs, integrating the confidence
equipped with Intel 3805U 1.9GHz CPU and 4GB RAM.
values from neural network predictions with those of
Moreover, all the models that constitute the proposed
Decision Trees and Random Forests, notably improves
IDS have been run 1000 times using different train and
the performance of the whole IDS. Finally, it is worth
test sets at every execution.
noticing that our system’s architecture facilitates paral-
The numerous tests performed on the system have chine learning models. Indeed, ignoring the phenomenon
demonstrated its reliability and accuracy in detecting of concept drift, like many current IDSs do, inevitably
malicious traffic, as well as its time efficiency. The IDS lead to performance degradation over time.
is able to recognize and identify 9 different types of at- Our future approach will try to overcome these chal-
tack in real-time, promptly alerting administrators to lenges by orchestrating supervised and unsupervised
minimize serious consequences. In fact, on average, the systems to exploit the benefits of both approaches. The
system misses attacks in very small percentages (close to detection of unknown attacks can rely on online unsu-
1%), while it requires extremely low execution time for pervised anomaly detection systems that are adept at
both the first and second levels: some slight difference recognizing signs of zero-day attacks, all the while auto-
is appreciated in dependence on the model used in the matically adapting to concept drift without the constant
ensemble. need for manual intervention. This, in turn, can also
Besides the good performance achieved, numerous reduce the frequency of model re-training and enhance
improvements are needed to address other important system efficiency. Such systems will be used in conjunc-
limitations, that are common to many IDSs in the litera- tion with supervised ones to improve the overall accuracy
ture. for known attacks.
First of all, the solutions proposed in the literature (as The efficacy of our methodologies will be validated
well as [10]) select the set of classifiers to be adopted through extensive experimental evaluation, showcasing
through a trial-and-error process and lack a formalized our system’s capability of real-time threat detection com-
methodology that can drive the design process in differ- pared to traditional models. This will provide the re-
ent scenarios. Moreover, many of the existing solutions search community with valuable insights into the effec-
have been designed ignoring the outbreak of unknown tiveness of different ML methods and ensemble strategies
attacks. Such a “closed-world” approach makes IDSs un- against a wide range of security attacks.
suitable for recognizing special types of attacks known Looking forward, we envision further enriching our
as “zero-day”. IDS framework to improve its resilience against unknown
attacks and concept drift, offering robust defenses against
the ever-evolving landscape of cyber threats.
5. Challenges and Goals
The main goal of the research unit is the design and 6. Research Unit
development of a novel class of IDSs based on the com-
bination of several dynamically orchestrated classifiers The Networks and Distributed Systems and Artificial Intel-
(both supervised and unsupervised), with the aim of rec- ligence research laboratories at the University of Palermo,
ognizing a large set of different threats, also detecting directed by Prof. Giuseppe Lo Re and Salvatore Gaglio,
the occurrence of zero-day attacks. have experience in several research fields such as dis-
Given the strong characterization of the many appli- tributed systems, cybersecurity, artificial intelligence,
cation scenarios in which IDSs are needed, the design and machine learning. In particular, the research unit
of the system architecture will be guided by a formal- has developed deep expertise in several topics related to
ized, rigorous, and replicable approach that can steer the cybersecurity domain that mainly concern the adop-
the realization of specific IDS instances. The goal is to tion of artificial intelligence to assist the detection and
design a scalable and modular architecture, capable of identification of potential threats in cyberspace. The iden-
maintaining a low computing load while guaranteeing tified methodologies and proposed solutions have been
high detection performance and responsiveness, even in applied in different scenarios, such as intrusion detection
the presence of huge amounts of data. systems [10], malware detection systems [13, 14], social
The main challenge will be the definition of adaptive network security [15, 16], privacy-preserving distributed
orchestration techniques, which will be crucial for the systems [17, 18], adversarial machine learning [19] and
design of IDSs capable of dynamically adjusting their secure crowdsensing [20].
ensemble strategies based on the observed context. This Furthermore, it is worth noting that the research
will include the integration of both supervised and un- group’s experience in applying artificial intelligence ap-
supervised learning approaches, allowing an adaptive proaches and methods to distributed systems and cy-
response to emerging threats. bersecurity challenges has been leveraged in several
To reach this ambitious goal, the system will also have funded research projects, such as FRASI - FRamework
to address the phenomenon of concept drift, which is the for Agent-based Semantic- aware In-teroperability (FAR
continuous shift of the statistical distribution of network MIUR D.M. 8 agosto 2000), Bigger Data (D.D. MIUR n.
data over time. This poses a big challenge for current 2690 dell’11.12.2013, Piano di Azione e Coesione), SeN-
IDSs, often necessitating manual retraining of their ma- Sori - SEnsor Node as a Service for hOme and buildings
eneRgy savIng (Industria 2015: Bando Nuove Tecnolo- [8] S. Seth, K. K. Chahal, G. Singh, A novel ensem-
gie per il Made in Italy), Smart Buildings - An Ambient ble framework for an intelligent intrusion detec-
Intelligence system for optimizing energy resources in tion system, IEEE Access 9 (2021) 138451–138467.
building complexes (PO FESR Sicilia 2007-2013), OnSi- doi:10.1109/ACCESS.2021.3116219.
cily.com - a Web 3.0 platform with intelligent virtual [9] X. Li, M. Zhu, L. T. Yang, M. Xu, Z. Ma, C. Zhong,
A.V.I. assistance (PO FESR Sicilia 2007-2013), VASARI - H. Li, Y. Xiang, Sustainable ensemble learning driv-
VAlorizzazione Smart del patrimonio ARtistico delle città ing intrusion detection model, IEEE Transactions
Italiane (PNR 2015-2020), CrowdSense (PO FESR Sicilia on Dependable and Secure Computing 18 (2021)
2014-2020), Smart Wave (PO FESR Sicilia 2014-2020), S6 1591–1604. doi:10.1109/TDSC.2021.3066202.
Project - A Smart, Social and SDN-based Surveillance [10] V. Agate, D. Felice Maria, A. De Paola, P. Ferraro,
System for Smart-cities (PO FESR Sicilia 2014-2020), S3 G. Lo Re, M. Morana, A behavior-based intrusion de-
Campus - SHARING, SMART AND SUSTAINABLE CAM- tection system using ensemble learning techniques.,
PUS (POC Sicilia 2014-2020 ), Smart Venues for Agrotech in: ITASEC, 2022, pp. 207–218.
Ecosystem (POC Sicilia 2014-2020). [11] A. De Paola, P. Ferraro, S. Gaglio, G. Lo Re,
M. Morana, M. Ortolani, D. Peri, A context-aware
system for ambient assisted living, in: S. F. Ochoa,
References P. Singh, J. Bravo (Eds.), Ubiquitous Computing
and Ambient Intelligence, Springer International
[1] T. Zoppi, A. Ceccarelli, T. Puccetti, A. Bondavalli,
Publishing, Cham, 2017, pp. 426–438.
Which algorithm can detect unknown attacks?
[12] I. Sharafaldin, A. H. Lashkari, A. A. Ghorbani, To-
comparison of supervised, unsupervised and meta-
ward generating a new intrusion detection dataset
learning algorithms for intrusion detection, Com-
and intrusion traffic characterization, in: Proceed-
puters & Security 127 (2023) 103107.
ings of the 4th International Conference on Infor-
[2] A. Khraisat, I. Gondal, P. Vamplew, J. Kamruzzaman,
mation Systems Security and Privacy - Volume 1:
Survey of intrusion detection systems: techniques,
ICISSP„ INSTICC, SciTePress, 2018, pp. 108–116.
datasets and challenges, Cybersecurity 2 (2019)
doi:10.5220/0006639801080116.
20. URL: https://doi.org/10.1186/s42400-019-0038-7.
[13] A. De Paola, S. Gaglio, G. Lo Re, M. Morana, A hy-
doi:10.1186/s42400-019-0038-7.
brid system for malware detection on big data, in:
[3] A. L. Buczak, E. Guven, A survey of data min-
IEEE INFOCOM 2018 - IEEE Conference on Com-
ing and machine learning methods for cyber se-
puter Communications Workshops (INFOCOM WK-
curity intrusion detection, IEEE Communications
SHPS), 2018, pp. 45–50. doi:10.1109/INFCOMW.
Surveys & Tutorials 18 (2016) 1153–1176. doi:10.
2018.8406963.
1109/COMST.2015.2494502.
[14] F. Concone, A. De Paola, G. Lo Re, M. Morana, Twit-
[4] J. Lu, A. Liu, F. Dong, F. Gu, J. Gama, G. Zhang,
ter analysis for real-time malware discovery, in:
Learning under concept drift: A review, IEEE Trans-
2017 AEIT International Annual Conference (2017
actions on Knowledge and Data Engineering 31
AEIT), Cagliari, Italy, 2017.
(2018) 2346–2363.
[15] F. Concone, G. Lo Re, M. Morana, S. K. Das, Spade:
[5] A. A. Aburomman, M. B. I. Reaz, A survey of intru-
Multi-stage spam account detection for online so-
sion detection systems based on ensemble and hy-
cial networks, IEEE Transactions on Dependable
brid classifiers, Computers & Security 65 (2017) 135–
and Secure Computing (2022) 1–16. doi:10.1109/
152. URL: https://www.sciencedirect.com/science/
TDSC.2022.3198830.
article/pii/S0167404816301572. doi:https://doi.
[16] F. Concone, G. Lo Re, M. Morana, C. Ruocco, Twitter
org/10.1016/j.cose.2016.11.004.
spam account detection by effective labeling, in:
[6] B. A. Tama, M. Comuzzi, K.-H. Rhee, Tse-ids: A two-
3rd Italian Conference on Cyber Security, ITASEC
stage classifier ensemble for intelligent anomaly-
2019, volume 2315, IT, 2019.
based intrusion detection system, IEEE Access 7
[17] V. Agate, P. Ferraro, G. Lo Re, S. K. Das,
(2019) 94497–94507. doi:10.1109/ACCESS.2019.
Blind: A privacy preserving truth discovery
2928048.
system for mobile crowdsensing, Journal of
[7] Y. Zhou, G. Cheng, S. Jiang, M. Dai, Building
Network and Computer Applications (2023)
an efficient intrusion detection system based
103811. URL: https://www.sciencedirect.com/
on feature selection and ensemble classifier,
science/article/pii/S1084804523002308. doi:https:
Computer Networks 174 (2020) 107247. URL:
//doi.org/10.1016/j.jnca.2023.103811.
https://www.sciencedirect.com/science/article/pii/
[18] V. Agate, A. De Paola, P. Ferraro, G. Lo Re,
S1389128619314203. doi:https://doi.org/10.
M. Morana, Secureballot: A secure open
1016/j.comnet.2020.107247.
source e-voting system, Journal of Network
and Computer Applications 191 (2021) 103165.
URL: https://www.sciencedirect.com/science/
article/pii/S1084804521001776. doi:https:
//doi.org/10.1016/j.jnca.2021.103165.
[19] S. Gaglio, A. Giammanco, G. Lo Re, M. Morana,
Adversarial machine learning in e-health: attack-
ing a smart prescription system, in: International
Conference of the Italian Association for Artificial
Intelligence (2021 AI*IA), Milan, Italy, 2021.
[20] F. Concone, G. Lo Re, M. Morana, Smcp: a
secure mobile crowdsensing protocol for fog-
based applications, Human-centric Computing
and Information Sciences 10 (2020) 1–23. URL:
https://doi.org/10.1186/s13673-020-00232-y. doi:10.
1186/s13673-020-00232-y.