=Paper=
{{Paper
|id=Vol-3762/529
|storemode=property
|title=Enhancing Cyber-threat detection coupling Deep Neural Ensemble Learning with XAI
|pdfUrl=https://ceur-ws.org/Vol-3762/529.pdf
|volume=Vol-3762
|authors=Malik Al-Essa,Giuseppina Andresini,Annalisa Appice,Donato Malerba
|dblpUrl=https://dblp.org/rec/conf/ital-ia/Al-EssaAAM24
}}
==Enhancing Cyber-threat detection coupling Deep Neural Ensemble Learning with XAI==
https://ceur-ws.org/Vol-3762/529.pdf
Enhancing Cyber-threat detection coupling Deep Neural
Ensemble Learning with XAI
Malik Al-Essa1,*,† , Giuseppina Andresini1,2,*,† , Annalisa Appice1,2,† and Donato Malerba1,2,†
1
University of Bari Aldo Moro, Bari, Italy
2
Consorzio Interuniversitario Nazionale per l’Informatica - CINI, Bari, Italy
Abstract
In the digital age, the use of deep learning is one of the most powerful machine learning paradigms for cybersecurity.
Despite the amazing results recently achieved with deep learning methods in securing the digital infrastructures of modern
organizations, the security of neural models can easily be jeopardized by adversarial attacks. This article describes a recently
published cyber-threat detection method, named PANACEA, that combines Adversarial Training and eXplainable Artificial
Intelligence (XAI) to increase the diversity of multiple neural models fused together through a neural ensemble system.
Experiments carried out on several benchmark cybersecurity datasets show the beneficial effects of the proposed combination
of Adversarial Training, Ensemble Learning and XAI on the accuracy of multi-class classifications of cyber-data achieved by
the neural method.
Keywords
Ensemble Learning, Adversarial Training, eXplainable Artificial Intelligence, Cyber-threat Detection
1. Introduction multiple samples produced in the same situation. Hence,
an accurate ensemble system may be produced through
During the last decade, the cybersecurity literature has the fusion of base models that perform decisions which
conferred a high-level role in deep learning as a pow- give more importance to different sub-areas of the in-
erful learning paradigm to detect ever-evolving cyber- put feature space. For this purpose, we use the XAI
threats in modern security systems. In particular, recent DALEX framework [5] to explain the global feature im-
cybersecurity studies have shown that deep learning per- portance in neural models. Specifically, we adopt a com-
formance can be further strengthened with ensemble bination of XAI and clustering to select ensemble base
learning systems [1] that are able to obtain better gen- models that achieve high explanation diversity. Finally,
eralization by reducing the dispersion of predictions of we use a multi-headed neural network architecture that
single models and gaining model accuracy. However, se- fine-tunes simultaneously base neural models selected
lecting the ensemble member models based on the local through DALEX-based clustering, by taking advantage of
model accuracy may lead to the issue of excessive en- a back-propagation strategy to share knowledge among
semble because the performance of the ensemble system multiple base models incorporated as sub-network blocks
may not be significantly improved by some of the se- in the ensemble system.
lected models. Therefore, several scholars encourage the Motivations for adopting this neural ensemble method
diversity among individual models of deep ensembles, in in cybersecurity problems can be mainly founded in the
addition to the accuracy of individual models, to learn peculiarities of the network intrusion detection problems,
diverse aspects of training data [2]. where samples of different attack families commonly
In [3, 4], we have recently proposed a new XAI-based have signatures involving different features. For exam-
method, named PANACEA, that is mainly founded on the ple, as illustrated by [6], “the time between the SYN ACK
idea that different sub-areas of the input feature space and the ACK response” is relevant for detecting shell-
can be equally relevant to achieve a correct decision for code intrusions, while it becomes less important when
detecting other types of attacks. Shellcode, in fact, is an
Ital-IA 2024: 4th National Conference on Artificial Intelligence, orga-
exploiting attack in which the attacker penetrates a piece
nized by CINI, May 29-30, 2024, Naples, Italy
*
Corresponding author. of code from a shell to control a target machine using the
†
These authors contributed equally. standard TCP/IP socket connections.
$ malik.alessa@uniba.it (M. Al-Essa); Based upon these considerations, our point of view
giuseppina.andresini@uniba.it (G. Andresini); is that being able to fuse deep neural models that give
annalisa.appice@uniba.it (A. Appice); donato.malerba@uniba.it relevance to different network traffic feature signatures
(D. Malerba)
(and, consequently, input feature sub-spaces) may help
0000-0002-0892-975X (M. Al-Essa); 0000-0002-5272-644X
(G. Andresini); 0000-0001-9840-844X (A. Appice); in improving the accuracy of a multi-class deep neural
0000-0001-8432-4608 (D. Malerba) ensemble trained to recognize different cyber-attack pat-
© 2024 Copyright for this paper by its authors. Use permitted under Creative Commons License
Attribution 4.0 International (CC BY 4.0). terns such as various categories of network traffic intru-
CEUR
ceur-ws.org
Workshop ISSN 1613-0073
Proceedings
Figure 1: Schema of PANACEA
sions. Our argument is mainly supported by experiments • The generation of an adversarial set 𝒜 produced
performed with three benchmark network intrusion de- by 𝒟 with data perturbation threshold 𝜖 by using
tection datasets, namely NSL-KDD, UNSW- NB15 and CI- 𝑀𝜃 . The adversarial samples are produced using
CIDS17, that comprise multiple real categories of network the FGSM algorithm.
traffic intrusions (comprising rare attacks). In addition, • The training of 𝜂 neural model candidates learned
to explore the adaptability of the proposed method to from 𝒟, augmented with subsets of 𝜎 adversarial
other cyber-threat detection problems, we also evaluated samples randomly selected from 𝒜.
the effectiveness proposed method in a benchmark mal- • The use of a post-hoc global XAI technique,
ware detection problem, namely CICMalDroid20, since namely DALEX, to explain the decisions of neural
we expect that, similarly to network traffic intrusions, model candidates and generate a feature-vector
different malware categories may have diverse feature explanation of each neural model candidate.
signatures. • A clustering stage (𝑘-medoids method) to group
This paper summarises some of the main results pub- neural model candidates with similar feature ex-
lished in [3, 4]. The PANACEA method is presented in planation vectors in the same clusters, and neural
Section 2. Section 3 illustrates the main results achieved model candidates with dissimilar feature expla-
in the evaluation of the proposed method. Finally, Sec- nation vectors in separate clusters. Since each
tion 4 draws conclusions and sketches future research cluster medoid is a neural model candidate that
directions. acts as the cluster’s prototype, 𝑘 medoids (chosen
using the Elbow method) are selected as the base
2. PANACEA method neural models for the ensemble fusion.
• A multi-headed neural network that fuses to-
Let us consider a dataset 𝒟 = {(x𝑖 , 𝑦𝑖 )}𝑁𝑖=1 of 𝑁 train- gether base neural models selected through clus-
ing samples, where x ∈ R𝑑 is a 𝑑-dimensional vector tering.
of input features that describe cyber-data samples, and
Notice that the performance of PANACEA may depend
𝑦 ∈ {1, . . . , 𝐾} is the label variable with 𝐾 classes
on the input parameters:(1) 𝜖 that represents the amount
(benign class and several categories of cyber-threats), ac-
of data perturbation considered to generate adversarial
cording to labels of samples historically collected.
samples; (2) 𝜎 that defines the number of adversarial sam-
The PANACEA method, illustrated in Figure 1, is based
ples randomly selected for learning each neural model
on the following steps:
candidate with the adversarial training strategy; (3) 𝜂
• The training of an initial neural model that is the number of distinct neural model candidates
𝑀𝜃 : R𝑑 ↦→ 𝑌 with parameter 𝜃 learned from 𝒟. learned with the adversarial training strategy. In general,
the perturbation 𝜖 is selected as a small value in the range BASELINE also in this configuration. In addition, there is
between 0 and 0.1 [7], to scale the noise and ensure that at least one tested configuration of PANACEA that outper-
perturbations are small enough to remain undetected to forms BASELINE in NSL-KDD. Finally, also in NSL-KDD
the human eye, but large enough to fool the attacked the gain in accuracy is observed along WeightedF1 and
neural model. In PANACEA the value of 𝜖 is automati- OA, but not along MacroF1. This is due to the presence
cally selected based on the characteristics of adversarial of minority classes in both NSL-KDD and UNSW-NB15.
samples. This is based on the idea that the value at which In fact, in both datasets, the ensemble strategy allows us
a lower 𝜖 stops perturbing training samples, by dimin- to gain accuracy by better classifying samples of major-
ishing the number of misclassified adversarial training ity classes, while we may lose accuracy by classifying
samples, may correspond to an adequate value of 𝜖 for samples of minority classes. This intuition is confirmed
gaining accuracy with the adversarial training strategy. by the analysis of detailed F1 per class, reported [4]. No-
Based on this idea, for each 𝜖 in the range [0, 0.1], the tably [4] also reports an extensive analysis of the accuracy
adversarial set 𝒜𝜖 , produced from the original training performance of PANACEA compared to several, recent
set with initial neural model 𝑀𝜃 as target model, is con- state-of-the-art competitors, as well as the analysis of the
sidered. The Overall Accuracy (OA) of 𝑀𝜃 is computed accuracy performance achieved by PANACEA by using
on each 𝒜𝜖 and the Elbow method is used to pick the PGD, DeepFool and LowProFool in place of FGSM.
knee of the OA(𝒜𝜖 ) curve as the value of 𝜖. Notably, this To examine in-depth diversity, Figure 2 depicts the top-
procedure for the automatic selection of 𝜖 is independent 15 relevant features on the global decisions of the base
of both 𝜎 and 𝜂 that remain user-defined parameters neural models selected in NSL-KDD. Feature ranking
maps show how diverse input features play prominent
roles in explaining the decisions of the base neural mod-
3. Evaluation study els selected for the ensemble fusion in PANACEA. For
example, the input feature “serror_rate", that is ranked
Four benchmark multi-class datasets, i.e., NSL-KDD,
in third place for the neural model medoids of clusters
UNSW-NB15, CICIDS17 (network security datasets) and
2, 3 and 7 of NSL-KDD, is not even in the top-15 for
CICMalDroid20 (malware security dataset) were consid-
the medoid of cluster 6. Notably, humans may inspect
ered to evaluate the performance of PANACEA. Exper-
this explanation result to confirm the selection of neural
iments were conducted by dividing each dataset into
model candidates automatically selected by PANACEA or
training set and testing set. The detailed description of
perform a manual update of the automatic selection (with
the experimental set-up is reported in [4].
model deletions or additions) according to background
The most of experiments were conducted with 𝜎 = 5%
knowledge.
and 10% of the training set size, considering the values
We complete this article by illustrating an example
of elbow 𝜖 automatically selected with the Elbow method
that shows how the ensemble model of PANACEA gains
and fixing 𝜂 = 100 for all datasets. However, further
accuracy in a cyber-threat detection task compared to
experiments exploring the sensitivity of the performance
the single model of BASELINE. For this purpose, we con-
of PANACEA to the number of models 𝜂 are illustrated
sider an R2L sample of the test set of NSL-KDD that
in [4].
was wrongly classified by BASELINE in the class Nor-
Table 1 reports the number of neural models (𝑘) that
mal, while it was correctly recognised in the class R2L
the clustering step of PANACEA selected for the ensem-
by PANACEA. We analyse this sample by using SHAP
ble fusion, as well as WeightedF1, MacroF1 and OA of
that is a local algorithm to measure the effect of an input
PANACEA in the considered experimental setting. All the
feature on the assignment of a sample to a class with a
accuracy metrics were measured on the testing set of each
neural model. Figure 3 shows the five most important
dataset. As BASELINE, we considered the deep neural net-
input features identified by SHAP to see the sample in the
work that was trained in the first step of PANACEA as the
class R2L with the models learned by both BASELINE and
initial neural model for the adversarial sample production.
PANACEA. Let us consider that only PANACEA predicted
We recall that the number of clusters 𝑘 was automatically
this sample in the class R2L.
identified during the clustering step of PANACEA. The
Both BASELINE and PANACEA share the same
results show that PANACEA outperforms BASELINE, in-
top-3 features, i.e., service_http, service_ftp_data and
dependently of the number 𝜎 of adversarial samples pro-
dst_host_srv_count. Notably, these three features are
cessed in UNSW-NB15, CICIDS17 and CICMalDroid20.
recognised as important to detect R2L attacks also in
In these three datasets, the gain in accuracy is commonly
[8]. The input feature in the fourth place of the feature
observed equally along WeightedF1, MacroF1 and OA.
ranking of PANACEA is protocol_type_tcp that does not
The only exception is the MacroF1 of PANACEA with
appear in the feature ranking of BASELINE. The authors
𝜎 = 5% in UNSW-NB15. However, both WeightedF1
of [9] report that the simultaneous use of the TCP proto-
and OA of PANACEA outperform WeightedF1 and OA of
col and the FTP service is to be considered a symptom of
described a deep learning method for multi-class clas-
sification of cyber-data.1 The proposed method trains
an ensemble of base neural models, whose weights are
initialised with an adversarial training strategy. We use
an XAI-based approach to increase the diversity of the
neural models selected to be fused together through the
ensemble system.
Notably, this article delves into one of the current
research directions carried out by Laboratory KDDE
(https://kdde.di.uniba.it/) at the University of Bari "Aldo
Moro", which aims at exploring a Symbiotic AI approach
to Cybersecurity. The team has recently published sev-
eral papers in this field (e.g., [10, 11, 6, 12, 13]). In partic-
ular, the newest studies [3, 4] stay under the umbrella of
Symbiotic AI, as they explore how Explainability of AI
systems can be leveraged as a valuable means to allow
deep neural models to gain accuracy under critical con-
ditions commonly occurring in cybersecurity problems,
e.g., class imbalance, attack signature diversity. They
provide a mechanism that can explain to humans how
the candidate models are selected for ensemble systems.
Figure 2: Top-15 feature ranking map of the base neural
On the other side, these studies stay under the umbrella
models selected through the clustering step of PANACEA in
of Cybersecurity, as XAI is used to improve the perfor-
NSL-KDD
mance of a cyber-threat detection ensemble model on
multiple attack categories by allowing us to identify and
use the multiple input sub-space that can help in detect-
a possible Warez Master attack in network traffic. Warez ing attacks with diverse signature. In addition, the use
Master is a subcategory of R2L attacks, where attackers of XAI tools allows us to perform a step forward to gain
exploit a system bug associated with FTP to send packets the trust of stakeholders in AI decisions. In fact, it allows
of illegal software to a target host [9]. We note that FTP us to disclose cyber-data patterns that are hidden in how
is a service based on the TCP protocol. Therefore, this the AI models achieve a decision and explain why a black
example shows how the ensemble model of PANACEA box model can actually achieve higher performance than
manages to bring out the existence of feature patterns another one in cyber-threat detection.
useful for the recognition of attack classes that are often By continuing along this research direction, the team
ignored by the single model of BASELINE. These conclu- is working on the use of XAI to examine and explain
sions are also supported by the study of [8], that identifies the evasion ability of state-of-the-art attack methods for-
both service_ftp_data and protocol_type_tcp features as mulated for Windows PE malware detection problems.
the most important features to detect R2L attacks. In ad- In addition, the team is investigating emerging learning
dition, BASELINE, differently from PANACEA, identifies frameworks (such as distillation) to leverage explanations
serror_rate as one of the most relevant features for recog- disclosed through attention layers to improve the per-
nizing the sample as an R2L attack. However, neither [8] formance of deep neural models trained for cyber-threat
nor [9] identify this feature as one of the most prominent detection.
features for this type of attack.
In short, the emergence of protocol_type_tcp can be
considered as an important input feature instead of ser- 5. Acknowledgments
ror_rate motivates the ability of PANACEA in correctly
recognising the considered R2L sample and, in general, Malik AL-Essa is supported by PON RI 2014-2020 - Ma-
the ability of outperforming BASELINE in the recogni- chine Learning per l’Investigazione di Cyber-minacce
tion of R2L attacks (that passes from F1(R2L)=0.55 for e la Cyber-difesa - CUP H98B20000970007. Giuseppina
BASELINE to F1(R2L)=0.64 for PANACEA. Andresini is supported by the project FAIR - Future AI
Research (PE00000013), Spoke 6 - Symbiotic AI, under the
NRRP MUR program funded by the NextGenerationEU.
4. Conclusion
1
The original research illustrated in [4] was published under
In this paper, we have summarized the main results of Creative Commons License Attribution 4.0 (CC BY 4.0) https:
our newest research illustrated in [4], where we have //creativecommons.org/licenses/by/4.0/
(a) BASELINE (b) PANACEA
Figure 3: Top-5 input features considered by both BASELINE (3a) and PANACEA (3b) to recognize an R2L attack in the class
R2L
Table 1
WeightedF1, MacroF1 and OA of PANACEA with 𝜎 = 5% and 10% of the training set size and BASELINE. 𝑘 denotes the
number of distinct neural models automatically selected in the clustering step of PANACEA over 𝜂 = 100 neural model
candidates. The best results are in bold.
dataset method 𝑘 WeightedF1 MacroF1 OA
BASELINE - 0.80 0.64 0.80
NSL-KDD PANACEA (𝜎 = 5%) 8 0.79 0.60 0.80
PANACEA (𝜎 = 10%) 7 0.83 0.64 0.84
BASELINE - 0.74 0.42 0.74
UNSW-NB15 PANACEA (𝜎 = 5%) 12 0.77 0.41 0.77
PANACEA (𝜎 = 10%) 11 0.78 0.44 0.77
BASELINE - 0.92 0.64 0.91
CICIDS17 PANACEA (𝜎 = 5%) 9 0.98 0.73 0.97
PANACEA (𝜎 = 10%) 9 0.99 0.94 0.99
BASELINE - 0.83 0.80 0.83
CICMalDroid20 PANACEA (𝜎 = 5%) 13 0.90 0.88 0.89
PANACEA (𝜎 = 10%) 18 0.85 0.83 0.86
Annalisa Appice and Donato Malerba are partially sup- detection, Machine Learning (2024). doi:10.1007/
ported by project SERICS (PE00000014) under the NRRP s10994-023-06470-2.
MUR National Recovery and Resilience Plan funded by [5] P. Biecek, DALEX: Explainers for complex pre-
the European Union - NextGenerationEU. dictive models in R, Journal of Machine Learning
Research 19 (2018) 1–5.
[6] G. Andresini, A. Appice, F. P. Caforio, D. Malerba,
References G. Vessio, ROULETTE: A neural attention multi-
output model for explainable network intrusion
[1] B. A. Tama, S. Lim, Ensemble learning for intrusion
detection, Expert Systems with Applications (2022)
detection systems: A systematic mapping study and
117144. doi:10.1016/j.eswa.2022.117144.
cross-benchmark evaluation, Computer Science
[7] T. Bai, J. Luo, J. Zhao, B. Wen, Q. Wang, Recent
Review 39 (2021) 1–27. doi:10.1016/j.cosrev.
advances in adversarial training for adversarial ro-
2020.100357.
bustness, in: 30th International Joint Conference on
[2] X. Dong, Z. Yu, W. Cao, Y. Shi, Q. Ma, A sur-
Artificial Intelligence, IJCAI 2021, IJCAI.ORG, 2021,
vey on ensemble learning, Frontiers of Com-
pp. 4312–4321. doi:10.24963/ijcai.2021/591.
puter Science 14 (2020) 241–258. doi:10.1007/
[8] M. Sabhnani, G. Serpen, KDD feature set complaint
s11704-019-8208-z.
heuristic rules for R2L attack detection, in: Inter-
[3] M. Al-Essa, G. Andresini, A. Appice, D. Malerba,
national Conference on Security and Management,
Panacea: A neural model ensemble for cyber-threat
SAM 2003, CSREA Press, 2003, pp. 310–316.
detection, 2023. doi:10.1109/DSAA60987.2023.
[9] M. Wang, K. Zheng, Y. Yang, X. Wang, An explain-
10302518.
able machine learning framework for intrusion de-
[4] M. AL-Essa, G. Andresini, A. Appice, D. Malerba,
tection systems, IEEE Access 8 (2020) 73127–73141.
Panacea: a neural model ensemble for cyber-threat
doi:10.1109/ACCESS.2020.2988359.
[10] F. P. Caforio, G. Andresini, G. Vessio, A. Appice,
D. Malerba, Leveraging grad-cam to improve the
accuracy of network intrusion detection systems,
in: 24th Conference on Discovery Science , DS 2021,
volume 12986 of Lecture Notes in Computer Science,
Springer, 2021, pp. 385–400.
[11] G. Andresini, F. Pendlebury, F. Pierazzi, C. Loglisci,
A. Appice, L. Cavallaro, INSOMNIA: towards
concept-drift robustness in network intrusion de-
tection, in: 14th ACM Workshop on Artificial Intel-
ligence and Security, ACM, 2021, pp. 111–122.
[12] M. AL-Essa, G. Andresini, A. Appice, D. Malerba,
XAI to explore robustness of features in adver-
sarial training for cybersecurity, in: Founda-
tions of Intelligent Systems, Springer Interna-
tional Publishing, 2022, pp. 117–126. doi:10.1007/
978-3-031-16564-1_12.
[13] M. Al-Essa, G. Andresini, A. Appice, D. Malerba,
An XAI-based adversarial training approach for
cyber-threat detection, in: 2022 IEEE International
Conference on Cyber Science and Technology
Congress, CyberSciTech 2023, IEEE, 2022, pp. 1–8.
doi:10.1109/DASC/PiCom/CBDCom/Cy55231.
2022.9927842.
A. Online Resources
The source code of PANACEA implementation is available
online at https://github.com/malikalessa/PANACEA.