In many cybersecurity contexts, the real-time detections of hostile actions play a fundamental role in protecting network infrastructures. In this scenario, Intrusion Detection Systems (IDS), based on signature-based or anomaly detection, are widely used to analyze network trafic. The signature-based detection relies on databases of known attack signatures, and anomaly detection is mainly based on Artificial Intelligence (AI) techniques. The latter is promising to detect new kinds of cyberattacks in real time. In this work, we propose ReTiNA-IDS, a framework that integrates the CICFlowmeter tool with Machine Learning techniques to analyze Real-Time network trafic patterns and detect abnormalities that may suggest a possible intrusion. The considered machine learning techniques, random forest and multi-layer network, are based on selected features to enhance eficiency and scalability. To select the features and train the models, we use a version of the public dataset, CSECICI-IDS2018. The framework's efectiveness has been tested in real-case scenarios by identifying diferent forms of intrusion. Analyzing the results, we conclude that the proposed solution shows valuable features.
The use of machine learning approaches in intrusion
detection systems to obtain real-time analysis has been
exploited by many researchers. Many of them take
advantage of Deep Learning (DL) approaches. ARCADE is an
unsupervised DL-based approach for early anomaly
detection using 1D Convolutional Neural Networks (CNNs)
proposed by Lunardi et al. [
Kathareios et al. designed and tested a real-time
network AD system, able to operate on encrypted and non- 3.1. CICFlowMeter
encrypted network packets, based on two learning stages:
an autoencoder for adaptive unsupervised AD and a CICFlowmeter is a network trafic flow generator and
custom nearest-neighbour classifier to filter false pos- analyser [13, 14]. It generates bidirectional flows, where
itives [
The four architectures are a Deep Feed-Forward
Module (DFFM), a Stacked Restricted Boltzmann Machine
Module (SRBMM), and two recurrent modules, one
utilizing gated recurrent units (GRUM) and the other
utilizing long short-term memory (LSTMM). Catillo et al. [
In this section, we present the CICFlowMeter, an Ethernet trafic Bi-flow generator and analyzer for anomaly detection, and the Random Forest, a machine learning method used for classifying flow data and evaluating the importance of features. This classifier will then be integrated into CICFlowMeter for classifying network lfows.
The Random Forest is an ML ensemble model used for both classification and regression tasks. During training, the model creates numerous decision trees and determines the output class by either the mode (for classification) or the mean/average prediction (for regression) of the classes predicted by individual trees. Introduced by Breiman in [16], this approach combines the bagging technique with the random selection of features. Such a random selection ensures that the decision trees within the forest are uncorrelated. In the bagging phase, decision trees are constructed from bootstrap samples of the training dataset, where each sample is drawn with replacement, allowing for the possibility of repeated samples. These replicated datasets are then used to train decision trees, ensuring that each tree only sees diferent portions of the original dataset during training. This bagging approach is coupled with random feature selection, which involves using distinct random subsets of the en- Table 2 tire feature space to train each tree in the random forest. The first 13 attributes ordered by importance Usually, around √ features are employed in each split Id Attribute for a classification task that considers ′′ features. 1 FWD Init Win Bytes
The data used in this study is the CSE-CIC-IDS2018, a benchmark dataset for the evaluation of IDSs. Such data was collected by the Communications Security Establishment (CSE) and the Canadian Institute for Cybersecurity (CIC). The recorded data consists of ten days of trafic and includes seven types of attacks. Liu et al. identified some issues in such dataset related to the creation lifecycle, including attack orchestration, feature generation, documentation, and labelling and provided to reconstruct the datasets by deleting artefacts and corrected labelling logic, including corrected implementations of existing features and new features that capture valuable flow state information [17]. Table 1 reports the corrupt amount of data.
Bot Web - Brute Force Web Attack - XSS DoS Attacks DDoS Attacks FTP-Patator Infiltration SQL Injection SSH-Patator Corruption Rate (%)
We evaluate the performance and efectiveness of the approaches by using Precision ( ), Recall () and , defined as follows = =
ReTiNA-IDS, Real-Time anomaly Detection IDS Approach, integrates a ML model mainly based on Random Forest in the CICFlowMeter tool to detect Real-Time cyber-attacks and act as a simple IDS. The Random Forest classifier considers only 13 of the 80 features calculated by the CICFlowMeter tool. The list of features with the relative description, selected by another Random Forest model, is in Table 2. After being trained, the model has been exported in a pmml format with the use of the “sklearn-pmml-model“ library from Sklearn [18]. The exported model is then imported into CICFlowMeter, which is developed in Java.
The proposed approach is based on Random Forest, described in Section 3.2, and its scheme is shown in Figure 2. In this study, the used dataset is a revised version of CSE-CIC-IDS2018, as introduced in Section 3.3. The dataset consists of the network trafic captured on ten days, stored in 10 distinct files according to the day of data capture, as shown in Table 3.
The first step of the preprocessing consists of data cleaning, i.e., removing missing values, such as incomplete rows, and containing invalid (or infinite) numerical values. Moreover, many non-relevant features for spotting cyber-attacks have been eliminated, such as the IP address of the sender and receiver, the connection timestamp, the protocol type, and the destination/sender port. Furthermore, the trafic data related to Web Attacks is deleted since its volume is insuficient.
Class BENIGN DoS Attack BruteForce Attack PortScan Attack BotNet Attack Total
The collected data related to network trafic is substantially unbalanced: benign trafic is more prevalent than malicious trafic. To balance the data, we have used the one step of the bootstrapping procedure, implemented in the resample function of Sklearn. Due to the corrupted data on the original dataset, it does not contain data related to FTP Brute Force attacks. Therefore, we have added this kind of data by collecting such data during a Figure 3: Confusion Matrix of the Random Forest Classifier simulation of brute force attacks via FTP (File Transfer Protocol). The simulation involved the use of a Windows The performance of the model, evaluated in terms of host (victim machine) and a Kali-Linux host (attacker ma- Precision, Recall and 1-score, is shown in the Table 5. chine), both in the same local area network (connected to the same router). The victim machine runs a FileZilla server, an open-source software utility that facilitates the 5. Experimental Setup transmission of files using the File FTP. It enables users to establish their own FTP servers or connect to existing FTP servers to exchange data, and the victim machine accepts connections on port 21, used to attack. When the FileZilla server on the victim machine is running, the Kali Linux host performs a brute-force attack using Patator, a multi-purpose brute-forcer tool [19]. Table 4 shows the amount of data and the relative kind of attack, after the cleaning and balancing phases.
The ML models have been implemented in a Google Colab document with Python 3. The default CPU in the environment is an Intel Xeon CPU equipped with 2 virtual CPUs (vCPUs) and 13GB of RAM [20]. For this study, the configuration involved the utilization of extra RAM, resulting in a total memory capacity of 50GB (included with Google Colab Pro [20]).
For data handling, preprocessing, analysis, training, and evaluation metrics, the recommended model was built and evaluated using Numpy [21], Pandas [22], and 4.1.3. Feature Selection and Classifier Scikit Learn [23]. Matplotlib [24] were used to visualTo select the features, a Random Forest has been consid- ize the data. The testing phase for this study used a ered and implemented by setting up the depth of each Windows operating system for the with the following decision tree and number of estimators to 16 and 20, specifications: an Intel Core i5-4670 CPU at 3.40GHz, 16 GB of DDR4 memory and a Nvidia GTX 1050 Ti GPU.
Retina-IDS, a tool that integrates an ML model into CI- In this work, we have presented ReTiNA-IDS, a tool that CFlowMeter, analyzes data patterns and distinguishes integrates an ML model into CICFlowMeter, which anabenign trafic from malicious trafic. The testing phase of lyzes data patterns and distinguishes benign trafic from ReTiNA-IDS intends to assess the eficiency and eficacy malicious trafic in real-time. The ML model is based on of the machine learning model in real-world network a Random Forest, used to select features and to classify situations. We take advantage of the Graphical Network the data. The testing phase, performed by running the Simulator-3 (GNS3) software, an open-source network tool in a normal trafic situation (without performing simulation tool used for creating, modelling, and testing any cyberattack) in a local network and the University virtual and real networks [25], to perform the simulations. of Camerino’s network, shows that the tool does not To reach the aim, we create a simple network composed identify false positives. of a Cisco router [26] and two generic switches, outlining In the near future, we intend to test the approach in bottwo diferent areas of a hypothetical Local Area Network net trafic to investigate the performance of the ReTiNA(LAN), a Windows machine and a Kali Linux machine. IDS. To reach this aim, we intend to create a central server Figure 4shows the network infrastructure. to control potentially infected hosts. Moreover, we have planned to consider other machine learning models, both supervised and unsupervised. Moreover, motivated by the results obtained for modelling and verifying properties of Collective Adaptive Systems [27, 28, 29], we intend to define formal approaches to specify and verify properties of the data trafic to monitor the trafic and identify anomalous pattern in the trafic.
The Windows machine represents the hypothetical victim running the Retina-IDS tool, acting as an IDS, while the Kali Linux machine plays the role of attacker.
The victim machine is a Windows 10 host, while the used Kali Linux version is Kali 2023.4. Instead, the victim machine is a Windows 10 host.
Diferent attack simulations were performed, each one resulting in a positive detection by the tool: • DoS attacks • File Transfer Protocol (FTP) and Secure SHell
(SHH) Bruteforce attacks • Portscan attacks
Additionally, more tests were performed with the tool running in a normal trafic situation (without performing any cyberattack) in a local network and in the University of Camerino’s network, for a total of around 5 hours of workload. The purpose of letting the tool run for hours on end was to see whether any crashes occurred during execution and to spot any false positive results. During the experiments zero false positives were identified.
Acknowledgements. This work has been funded by the European Union - NextGenerationEU under the Italian Ministry of University and Research (MUR) National Innovation Ecosystem grant ECS00000041 - VITALITY CUP J13C22000430001 in: Web, Artificial Intelligence and Network Appli- cic-ids-2017 and cse-cic-ids-2018, in: 2022 IEEE cations: Proceedings of the Workshops of the 34th Conference on Communications and Network SeInternational Conference on Advanced Informa- curity (CNS), IEEE, 2022, pp. 254–262. tion Networking and Applications (WAINA-2020), [18] scikit-learn: machine learning in python — scikitSpringer, 2020, pp. 687–696. learn 1.4.1 documentation, 2024. URL: https:// [6] Q. R. S. Fitni, K. Ramli, Implementation of ensemble scikit-learn.org/stable/index.html. learning and feature selection for performance im- [19] Kali linux tools, patator, 2024. URL: https://www. provements in anomaly-based intrusion detection kali.org/tools/patator/. systems, in: 2020 IEEE International Conference [20] Google, Google colab, 2024. URL: https://research. on Industry 4.0, Artificial Intelligence, and Com- google.com/colaboratory/faq.html. munications Technology (IAICT), IEEE, 2020, pp. [21] C. R. Harris, K. J. Millman, S. J. van der Walt, R. Gom118–124. mers, P. Virtanen, D. Cournapeau, E. Wieser, J. Tay[7] W. W. Daniel, The spearman rank correlation coef- lor, S. Berg, N. J. Smith, R. Kern, M. Picus, S. Hoyer, ifcient, Biostatistics: A Foundation for Analysis in M. H. van Kerkwijk, M. Brett, A. Haldane, J. F. del the Health Sciences (1987). Río, M. Wiebe, P. Peterson, P. Gérard-Marchant, [8] S. Gamage, J. Samarabandu, Deep learning meth- K. Sheppard, T. Reddy, W. Weckesser, H. Abbasi, ods in network intrusion detection: A survey and C. Gohlke, T. E. Oliphant, Array programming an objective comparison, Journal of Network and with NumPy, Nature 585 (2020) 357–362. URL: Computer Applications 169 (2020) 102767. doi:10. https://doi.org/10.1038/s41586-020-2649-2. doi:10. 1016/j.jnca.2020.102767. 1038/s41586-020-2649-2. [9] G. Karatas Baydogmus, O. Demir, O. Sahingoz, [22] T. pandas development team, pandas-dev/pandas: Increasing the performance of machine learning- Pandas, 2020. URL: https://doi.org/10.5281/zenodo. based idss on an imbalanced and up-to-date dataset, 3509134. doi:10.5281/zenodo.3509134. IEEE Access PP (2020) 1–1. doi:10.1109/ACCESS. [23] F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, 2020.2973219. B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, [10] B. Jason, Smote for imbalanced classification with R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, python, 2021. D. Cournapeau, M. Brucher, M. Perrot, E. Duch[11] V. Kanimozhi, T. P. Jacob, Artificial intelligence esnay, Scikit-learn: Machine learning in Python, based network intrusion detection with hyper- Journal of Machine Learning Research 12 (2011) parameter optimization tuning on the realistic cy- 2825–2830. ber dataset cse-cic-ids2018 using cloud computing, [24] J. D. Hunter, Matplotlib: A 2d graphics environin: 2019 international conference on communica- ment, Computing in Science & Engineering 9 (2007) tion and signal processing (ICCSP), IEEE, 2019, pp. 90–95. doi:10.1109/MCSE.2007.55. 0033–0036. [25] S. Worldwide, Gns3 documentation, 2024. URL: [12] J. Kim, J. Kim, H. Kim, M. Shim, E. Choi, Cnn- https://docs.gns3.com/docs/. based network intrusion detection against denial- [26] Cisco 3600 series - cisco, 2015. URL: of-service attacks, Electronics 9 (2020) 916. https://www.cisco.com/c/en/us/td/docs/ios/ [13] A. H. Lashkari, G. D. Gil, M. S. I. Mamun, A. A. 12_2/12_2x/12_2xa/release/notes/rn3600xa.html.
Ghorbani, Characterization of tor trafic using time [27] M. Loreti, M. Quadrini, A spatial logic for simplicial based features, in: International Conference on In- models, Log. Methods Comput. Sci. 19 (2023). formation Systems Security and Privacy, volume 2, [28] N. Del Giudice, L. Matteucci, M. Quadrini, SciTePress, 2017, pp. 253–262. A. Rehman, M. Loreti, Sibilla: A tool for reason[14] G. Draper-Gil, A. H. Lashkari, M. S. I. Mamun, A. A. ing about collective systems, Science of Computer Ghorbani, Characterization of encrypted and vpn Programming (2024) 103095. trafic using time-related, in: Proceedings of the 2nd [29] N. D. Giudice, L. Matteucci, M. Quadrini, international conference on information systems A. Rehman, M. Loreti, Sibilla: A tool for reasoning security and privacy (ICISSP), 2016, pp. 407–414. about collective systems, in: Coordination Models [15] U. of New Brunswick | UNB, Applications | research and Languages - 24th IFIP WG 6.1 International | canadian institute for cybersecurity | unb, 2017. Conference, COORDINATION 2022, Held as Part URL: https://www.unb.ca/cic/research/applications. of the 17th International Federated Conference on html. Distributed Computing Techniques, DisCoTec 2022, [16] L. Breiman, Random forests, Machine learning 45 Lucca, Italy, June 13-17, 2022, Proceedings, 2022, pp.
(2001) 5–32. 92–98. doi:10.1007/978-3-031-08143-9\_6. [17] L. Liu, G. Engelen, T. Lynar, D. Essam, W. Joosen,
Error prevalence in nids datasets: A case study on