Enhancing Cyber-threat detection coupling Deep Neural Ensemble Learning with XAI Malik Al-Essa1,*,† , Giuseppina Andresini1,2,*,† , Annalisa Appice1,2,† and Donato Malerba1,2,† 1 University of Bari Aldo Moro, Bari, Italy 2 Consorzio Interuniversitario Nazionale per l’Informatica - CINI, Bari, Italy Abstract In the digital age, the use of deep learning is one of the most powerful machine learning paradigms for cybersecurity. Despite the amazing results recently achieved with deep learning methods in securing the digital infrastructures of modern organizations, the security of neural models can easily be jeopardized by adversarial attacks. This article describes a recently published cyber-threat detection method, named PANACEA, that combines Adversarial Training and eXplainable Artificial Intelligence (XAI) to increase the diversity of multiple neural models fused together through a neural ensemble system. Experiments carried out on several benchmark cybersecurity datasets show the beneficial effects of the proposed combination of Adversarial Training, Ensemble Learning and XAI on the accuracy of multi-class classifications of cyber-data achieved by the neural method. Keywords Ensemble Learning, Adversarial Training, eXplainable Artificial Intelligence, Cyber-threat Detection 1. Introduction multiple samples produced in the same situation. Hence, an accurate ensemble system may be produced through During the last decade, the cybersecurity literature has the fusion of base models that perform decisions which conferred a high-level role in deep learning as a pow- give more importance to different sub-areas of the in- erful learning paradigm to detect ever-evolving cyber- put feature space. For this purpose, we use the XAI threats in modern security systems. In particular, recent DALEX framework [5] to explain the global feature im- cybersecurity studies have shown that deep learning per- portance in neural models. Specifically, we adopt a com- formance can be further strengthened with ensemble bination of XAI and clustering to select ensemble base learning systems [1] that are able to obtain better gen- models that achieve high explanation diversity. Finally, eralization by reducing the dispersion of predictions of we use a multi-headed neural network architecture that single models and gaining model accuracy. However, se- fine-tunes simultaneously base neural models selected lecting the ensemble member models based on the local through DALEX-based clustering, by taking advantage of model accuracy may lead to the issue of excessive en- a back-propagation strategy to share knowledge among semble because the performance of the ensemble system multiple base models incorporated as sub-network blocks may not be significantly improved by some of the se- in the ensemble system. lected models. Therefore, several scholars encourage the Motivations for adopting this neural ensemble method diversity among individual models of deep ensembles, in in cybersecurity problems can be mainly founded in the addition to the accuracy of individual models, to learn peculiarities of the network intrusion detection problems, diverse aspects of training data [2]. where samples of different attack families commonly In [3, 4], we have recently proposed a new XAI-based have signatures involving different features. For exam- method, named PANACEA, that is mainly founded on the ple, as illustrated by [6], “the time between the SYN ACK idea that different sub-areas of the input feature space and the ACK response” is relevant for detecting shell- can be equally relevant to achieve a correct decision for code intrusions, while it becomes less important when detecting other types of attacks. Shellcode, in fact, is an Ital-IA 2024: 4th National Conference on Artificial Intelligence, orga- exploiting attack in which the attacker penetrates a piece nized by CINI, May 29-30, 2024, Naples, Italy * Corresponding author. of code from a shell to control a target machine using the † These authors contributed equally. standard TCP/IP socket connections. $ malik.alessa@uniba.it (M. Al-Essa); Based upon these considerations, our point of view giuseppina.andresini@uniba.it (G. Andresini); is that being able to fuse deep neural models that give annalisa.appice@uniba.it (A. Appice); donato.malerba@uniba.it relevance to different network traffic feature signatures (D. Malerba) (and, consequently, input feature sub-spaces) may help  0000-0002-0892-975X (M. Al-Essa); 0000-0002-5272-644X (G. Andresini); 0000-0001-9840-844X (A. Appice); in improving the accuracy of a multi-class deep neural 0000-0001-8432-4608 (D. Malerba) ensemble trained to recognize different cyber-attack pat- © 2024 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). terns such as various categories of network traffic intru- CEUR ceur-ws.org Workshop ISSN 1613-0073 Proceedings Figure 1: Schema of PANACEA sions. Our argument is mainly supported by experiments • The generation of an adversarial set 𝒜 produced performed with three benchmark network intrusion de- by 𝒟 with data perturbation threshold 𝜖 by using tection datasets, namely NSL-KDD, UNSW- NB15 and CI- 𝑀𝜃 . The adversarial samples are produced using CIDS17, that comprise multiple real categories of network the FGSM algorithm. traffic intrusions (comprising rare attacks). In addition, • The training of 𝜂 neural model candidates learned to explore the adaptability of the proposed method to from 𝒟, augmented with subsets of 𝜎 adversarial other cyber-threat detection problems, we also evaluated samples randomly selected from 𝒜. the effectiveness proposed method in a benchmark mal- • The use of a post-hoc global XAI technique, ware detection problem, namely CICMalDroid20, since namely DALEX, to explain the decisions of neural we expect that, similarly to network traffic intrusions, model candidates and generate a feature-vector different malware categories may have diverse feature explanation of each neural model candidate. signatures. • A clustering stage (𝑘-medoids method) to group This paper summarises some of the main results pub- neural model candidates with similar feature ex- lished in [3, 4]. The PANACEA method is presented in planation vectors in the same clusters, and neural Section 2. Section 3 illustrates the main results achieved model candidates with dissimilar feature expla- in the evaluation of the proposed method. Finally, Sec- nation vectors in separate clusters. Since each tion 4 draws conclusions and sketches future research cluster medoid is a neural model candidate that directions. acts as the cluster’s prototype, 𝑘 medoids (chosen using the Elbow method) are selected as the base 2. PANACEA method neural models for the ensemble fusion. • A multi-headed neural network that fuses to- Let us consider a dataset 𝒟 = {(x𝑖 , 𝑦𝑖 )}𝑁𝑖=1 of 𝑁 train- gether base neural models selected through clus- ing samples, where x ∈ R𝑑 is a 𝑑-dimensional vector tering. of input features that describe cyber-data samples, and Notice that the performance of PANACEA may depend 𝑦 ∈ {1, . . . , 𝐾} is the label variable with 𝐾 classes on the input parameters:(1) 𝜖 that represents the amount (benign class and several categories of cyber-threats), ac- of data perturbation considered to generate adversarial cording to labels of samples historically collected. samples; (2) 𝜎 that defines the number of adversarial sam- The PANACEA method, illustrated in Figure 1, is based ples randomly selected for learning each neural model on the following steps: candidate with the adversarial training strategy; (3) 𝜂 • The training of an initial neural model that is the number of distinct neural model candidates 𝑀𝜃 : R𝑑 ↦→ 𝑌 with parameter 𝜃 learned from 𝒟. learned with the adversarial training strategy. In general, the perturbation 𝜖 is selected as a small value in the range BASELINE also in this configuration. In addition, there is between 0 and 0.1 [7], to scale the noise and ensure that at least one tested configuration of PANACEA that outper- perturbations are small enough to remain undetected to forms BASELINE in NSL-KDD. Finally, also in NSL-KDD the human eye, but large enough to fool the attacked the gain in accuracy is observed along WeightedF1 and neural model. In PANACEA the value of 𝜖 is automati- OA, but not along MacroF1. This is due to the presence cally selected based on the characteristics of adversarial of minority classes in both NSL-KDD and UNSW-NB15. samples. This is based on the idea that the value at which In fact, in both datasets, the ensemble strategy allows us a lower 𝜖 stops perturbing training samples, by dimin- to gain accuracy by better classifying samples of major- ishing the number of misclassified adversarial training ity classes, while we may lose accuracy by classifying samples, may correspond to an adequate value of 𝜖 for samples of minority classes. This intuition is confirmed gaining accuracy with the adversarial training strategy. by the analysis of detailed F1 per class, reported [4]. No- Based on this idea, for each 𝜖 in the range [0, 0.1], the tably [4] also reports an extensive analysis of the accuracy adversarial set 𝒜𝜖 , produced from the original training performance of PANACEA compared to several, recent set with initial neural model 𝑀𝜃 as target model, is con- state-of-the-art competitors, as well as the analysis of the sidered. The Overall Accuracy (OA) of 𝑀𝜃 is computed accuracy performance achieved by PANACEA by using on each 𝒜𝜖 and the Elbow method is used to pick the PGD, DeepFool and LowProFool in place of FGSM. knee of the OA(𝒜𝜖 ) curve as the value of 𝜖. Notably, this To examine in-depth diversity, Figure 2 depicts the top- procedure for the automatic selection of 𝜖 is independent 15 relevant features on the global decisions of the base of both 𝜎 and 𝜂 that remain user-defined parameters neural models selected in NSL-KDD. Feature ranking maps show how diverse input features play prominent roles in explaining the decisions of the base neural mod- 3. Evaluation study els selected for the ensemble fusion in PANACEA. For example, the input feature “serror_rate", that is ranked Four benchmark multi-class datasets, i.e., NSL-KDD, in third place for the neural model medoids of clusters UNSW-NB15, CICIDS17 (network security datasets) and 2, 3 and 7 of NSL-KDD, is not even in the top-15 for CICMalDroid20 (malware security dataset) were consid- the medoid of cluster 6. Notably, humans may inspect ered to evaluate the performance of PANACEA. Exper- this explanation result to confirm the selection of neural iments were conducted by dividing each dataset into model candidates automatically selected by PANACEA or training set and testing set. The detailed description of perform a manual update of the automatic selection (with the experimental set-up is reported in [4]. model deletions or additions) according to background The most of experiments were conducted with 𝜎 = 5% knowledge. and 10% of the training set size, considering the values We complete this article by illustrating an example of elbow 𝜖 automatically selected with the Elbow method that shows how the ensemble model of PANACEA gains and fixing 𝜂 = 100 for all datasets. However, further accuracy in a cyber-threat detection task compared to experiments exploring the sensitivity of the performance the single model of BASELINE. For this purpose, we con- of PANACEA to the number of models 𝜂 are illustrated sider an R2L sample of the test set of NSL-KDD that in [4]. was wrongly classified by BASELINE in the class Nor- Table 1 reports the number of neural models (𝑘) that mal, while it was correctly recognised in the class R2L the clustering step of PANACEA selected for the ensem- by PANACEA. We analyse this sample by using SHAP ble fusion, as well as WeightedF1, MacroF1 and OA of that is a local algorithm to measure the effect of an input PANACEA in the considered experimental setting. All the feature on the assignment of a sample to a class with a accuracy metrics were measured on the testing set of each neural model. Figure 3 shows the five most important dataset. As BASELINE, we considered the deep neural net- input features identified by SHAP to see the sample in the work that was trained in the first step of PANACEA as the class R2L with the models learned by both BASELINE and initial neural model for the adversarial sample production. PANACEA. Let us consider that only PANACEA predicted We recall that the number of clusters 𝑘 was automatically this sample in the class R2L. identified during the clustering step of PANACEA. The Both BASELINE and PANACEA share the same results show that PANACEA outperforms BASELINE, in- top-3 features, i.e., service_http, service_ftp_data and dependently of the number 𝜎 of adversarial samples pro- dst_host_srv_count. Notably, these three features are cessed in UNSW-NB15, CICIDS17 and CICMalDroid20. recognised as important to detect R2L attacks also in In these three datasets, the gain in accuracy is commonly [8]. The input feature in the fourth place of the feature observed equally along WeightedF1, MacroF1 and OA. ranking of PANACEA is protocol_type_tcp that does not The only exception is the MacroF1 of PANACEA with appear in the feature ranking of BASELINE. The authors 𝜎 = 5% in UNSW-NB15. However, both WeightedF1 of [9] report that the simultaneous use of the TCP proto- and OA of PANACEA outperform WeightedF1 and OA of col and the FTP service is to be considered a symptom of described a deep learning method for multi-class clas- sification of cyber-data.1 The proposed method trains an ensemble of base neural models, whose weights are initialised with an adversarial training strategy. We use an XAI-based approach to increase the diversity of the neural models selected to be fused together through the ensemble system. Notably, this article delves into one of the current research directions carried out by Laboratory KDDE (https://kdde.di.uniba.it/) at the University of Bari "Aldo Moro", which aims at exploring a Symbiotic AI approach to Cybersecurity. The team has recently published sev- eral papers in this field (e.g., [10, 11, 6, 12, 13]). In partic- ular, the newest studies [3, 4] stay under the umbrella of Symbiotic AI, as they explore how Explainability of AI systems can be leveraged as a valuable means to allow deep neural models to gain accuracy under critical con- ditions commonly occurring in cybersecurity problems, e.g., class imbalance, attack signature diversity. They provide a mechanism that can explain to humans how the candidate models are selected for ensemble systems. Figure 2: Top-15 feature ranking map of the base neural On the other side, these studies stay under the umbrella models selected through the clustering step of PANACEA in of Cybersecurity, as XAI is used to improve the perfor- NSL-KDD mance of a cyber-threat detection ensemble model on multiple attack categories by allowing us to identify and use the multiple input sub-space that can help in detect- a possible Warez Master attack in network traffic. Warez ing attacks with diverse signature. In addition, the use Master is a subcategory of R2L attacks, where attackers of XAI tools allows us to perform a step forward to gain exploit a system bug associated with FTP to send packets the trust of stakeholders in AI decisions. In fact, it allows of illegal software to a target host [9]. We note that FTP us to disclose cyber-data patterns that are hidden in how is a service based on the TCP protocol. Therefore, this the AI models achieve a decision and explain why a black example shows how the ensemble model of PANACEA box model can actually achieve higher performance than manages to bring out the existence of feature patterns another one in cyber-threat detection. useful for the recognition of attack classes that are often By continuing along this research direction, the team ignored by the single model of BASELINE. These conclu- is working on the use of XAI to examine and explain sions are also supported by the study of [8], that identifies the evasion ability of state-of-the-art attack methods for- both service_ftp_data and protocol_type_tcp features as mulated for Windows PE malware detection problems. the most important features to detect R2L attacks. In ad- In addition, the team is investigating emerging learning dition, BASELINE, differently from PANACEA, identifies frameworks (such as distillation) to leverage explanations serror_rate as one of the most relevant features for recog- disclosed through attention layers to improve the per- nizing the sample as an R2L attack. However, neither [8] formance of deep neural models trained for cyber-threat nor [9] identify this feature as one of the most prominent detection. features for this type of attack. In short, the emergence of protocol_type_tcp can be considered as an important input feature instead of ser- 5. Acknowledgments ror_rate motivates the ability of PANACEA in correctly recognising the considered R2L sample and, in general, Malik AL-Essa is supported by PON RI 2014-2020 - Ma- the ability of outperforming BASELINE in the recogni- chine Learning per l’Investigazione di Cyber-minacce tion of R2L attacks (that passes from F1(R2L)=0.55 for e la Cyber-difesa - CUP H98B20000970007. Giuseppina BASELINE to F1(R2L)=0.64 for PANACEA. Andresini is supported by the project FAIR - Future AI Research (PE00000013), Spoke 6 - Symbiotic AI, under the NRRP MUR program funded by the NextGenerationEU. 4. Conclusion 1 The original research illustrated in [4] was published under In this paper, we have summarized the main results of Creative Commons License Attribution 4.0 (CC BY 4.0) https: our newest research illustrated in [4], where we have //creativecommons.org/licenses/by/4.0/ (a) BASELINE (b) PANACEA Figure 3: Top-5 input features considered by both BASELINE (3a) and PANACEA (3b) to recognize an R2L attack in the class R2L Table 1 WeightedF1, MacroF1 and OA of PANACEA with 𝜎 = 5% and 10% of the training set size and BASELINE. 𝑘 denotes the number of distinct neural models automatically selected in the clustering step of PANACEA over 𝜂 = 100 neural model candidates. The best results are in bold. dataset method 𝑘 WeightedF1 MacroF1 OA BASELINE - 0.80 0.64 0.80 NSL-KDD PANACEA (𝜎 = 5%) 8 0.79 0.60 0.80 PANACEA (𝜎 = 10%) 7 0.83 0.64 0.84 BASELINE - 0.74 0.42 0.74 UNSW-NB15 PANACEA (𝜎 = 5%) 12 0.77 0.41 0.77 PANACEA (𝜎 = 10%) 11 0.78 0.44 0.77 BASELINE - 0.92 0.64 0.91 CICIDS17 PANACEA (𝜎 = 5%) 9 0.98 0.73 0.97 PANACEA (𝜎 = 10%) 9 0.99 0.94 0.99 BASELINE - 0.83 0.80 0.83 CICMalDroid20 PANACEA (𝜎 = 5%) 13 0.90 0.88 0.89 PANACEA (𝜎 = 10%) 18 0.85 0.83 0.86 Annalisa Appice and Donato Malerba are partially sup- detection, Machine Learning (2024). doi:10.1007/ ported by project SERICS (PE00000014) under the NRRP s10994-023-06470-2. MUR National Recovery and Resilience Plan funded by [5] P. Biecek, DALEX: Explainers for complex pre- the European Union - NextGenerationEU. dictive models in R, Journal of Machine Learning Research 19 (2018) 1–5. [6] G. Andresini, A. Appice, F. P. Caforio, D. Malerba, References G. Vessio, ROULETTE: A neural attention multi- output model for explainable network intrusion [1] B. A. Tama, S. Lim, Ensemble learning for intrusion detection, Expert Systems with Applications (2022) detection systems: A systematic mapping study and 117144. doi:10.1016/j.eswa.2022.117144. cross-benchmark evaluation, Computer Science [7] T. Bai, J. Luo, J. Zhao, B. Wen, Q. Wang, Recent Review 39 (2021) 1–27. doi:10.1016/j.cosrev. advances in adversarial training for adversarial ro- 2020.100357. bustness, in: 30th International Joint Conference on [2] X. Dong, Z. Yu, W. Cao, Y. Shi, Q. Ma, A sur- Artificial Intelligence, IJCAI 2021, IJCAI.ORG, 2021, vey on ensemble learning, Frontiers of Com- pp. 4312–4321. doi:10.24963/ijcai.2021/591. puter Science 14 (2020) 241–258. doi:10.1007/ [8] M. Sabhnani, G. Serpen, KDD feature set complaint s11704-019-8208-z. heuristic rules for R2L attack detection, in: Inter- [3] M. Al-Essa, G. Andresini, A. Appice, D. Malerba, national Conference on Security and Management, Panacea: A neural model ensemble for cyber-threat SAM 2003, CSREA Press, 2003, pp. 310–316. detection, 2023. doi:10.1109/DSAA60987.2023. [9] M. Wang, K. Zheng, Y. Yang, X. Wang, An explain- 10302518. able machine learning framework for intrusion de- [4] M. AL-Essa, G. Andresini, A. Appice, D. Malerba, tection systems, IEEE Access 8 (2020) 73127–73141. Panacea: a neural model ensemble for cyber-threat doi:10.1109/ACCESS.2020.2988359. [10] F. P. Caforio, G. Andresini, G. Vessio, A. Appice, D. Malerba, Leveraging grad-cam to improve the accuracy of network intrusion detection systems, in: 24th Conference on Discovery Science , DS 2021, volume 12986 of Lecture Notes in Computer Science, Springer, 2021, pp. 385–400. [11] G. Andresini, F. Pendlebury, F. Pierazzi, C. Loglisci, A. Appice, L. Cavallaro, INSOMNIA: towards concept-drift robustness in network intrusion de- tection, in: 14th ACM Workshop on Artificial Intel- ligence and Security, ACM, 2021, pp. 111–122. [12] M. AL-Essa, G. Andresini, A. Appice, D. Malerba, XAI to explore robustness of features in adver- sarial training for cybersecurity, in: Founda- tions of Intelligent Systems, Springer Interna- tional Publishing, 2022, pp. 117–126. doi:10.1007/ 978-3-031-16564-1_12. [13] M. Al-Essa, G. Andresini, A. Appice, D. Malerba, An XAI-based adversarial training approach for cyber-threat detection, in: 2022 IEEE International Conference on Cyber Science and Technology Congress, CyberSciTech 2023, IEEE, 2022, pp. 1–8. doi:10.1109/DASC/PiCom/CBDCom/Cy55231. 2022.9927842. A. Online Resources The source code of PANACEA implementation is available online at https://github.com/malikalessa/PANACEA.