Detection of DDoS Attacks with Gaussian Mixture Model

Abstract

A Distributed Denial of Service (DDoS) is an attack which aim is to stop or tamper with an online service incapacitating a server with a lfood of packages or requests, using internet or intranet. The main aim of the DDoS attack is to collapse the network or server with abnormal trafic to make the service unavailable for the legitimate users. This problem is particularly profound, due to the development of emerging technologies, such as cloud computing, the Internet of Things or artificial intelligence techniques, from which attackers can take advantage by launching a huge volume of DDoS attacks at a lower cost, and it is much harder to detect and prevent DDoS attacks, because DDoS trafic is similar to normal trafic. In this paper we implement a novel technique implementing an unsupervised Gaussian Mixture Model (GMM) based algorithm. Using a real trafic dataset, the CIC-DDoS2019, for benchmark, the proposed GMM can achieve recall, precision, and accuracy up to 99%. Experiments reveal that this can be a promising solution for the detection of DDoS attacks.

Keywords

• eol>Distributed Denial of Service (DDoS)
• Gaussian Mixture Model (GMM)
• Machine Learning
• Detection

1. Introduction

DoS (Denial of Service) attacks deplete the network bandwidth and computing resources of a targeted system by lfooding malicious trafic, preventing the target system from ofering regular services to legitimate users. DDoS (Distributed Denial of Service) goes even further on a much larger scale. DDoS attacks can take over a large number of compromised systems called bots, constituting a botnet, which are used to launch coordinated attacks on the victim system, from this kind of attack behavior, DDoS attacks can be devided in several branches as reported in [ 1 ]. Along with the emergence and advancement of Internet technologies [ 2 ], DDoS attacks are evolving and proliferating in scale, frequency, and sophistication. Organizations face potential threats to their network environment that may cause severe impacts to their operations, such as business downtime, data breaches, or even ransom demands from hackers [ 3 ]. The detection of DDoS attacks is essential before any mitigation approaches can be taken. In the early era, the alarm of DDoS attacks was triggered by rules programmed by trafic engineers, but in the current cybersecurity scenario, the application of artificial intelligence and, more specifically, machine learning (ML) ofers new and promising perspectives. Training predictive models to recognize anomalous patterns in network trafic provides a more agile and proactive means of detecting attacks. The Gaussian Mixture Model (GMM) is one such model, used for its ability to model complex distributions of data, such as those that characterize network trafic. In this paper, we propose a novel ML (Machine Learning) method based on GMM, for detecting DDoS malicious packets. The remaining part of this paper is organized as follows: Section II describes and analyze the related DDoS detection works with some proposed solutions. Section III describes and analyze the used dataset. Section IV explain all the steps that belongs to the workflow from the feature pre-processing to the GMM. Section V denotes the implementation of the project. Section VI resume the obtained results of the project also taking into account the phase of Cross-Validation. Finally, conclusions and future implementations are discussed in Section VII. Various ML technologies have been employed, mainly as classifiers, in the detection of DDoS attacks. Meng Wang et al. [ 4 ] proposed a dynamic multilayer perceptron (MLP) combined with a feature selection technique to detect DDoS attacks, where a feedback mechanism is applied to promote and reconstruct the detector system when detection is not accurate. In their model, as the complexities of trafic network increase and change, some of the selected features will not be able to distinguish the trafic and normal attacks and determine the failure therein. Nhu-Ngoc Dao et al. [ 5 ] proposed the approach of source based IP filtering technique to defeat DDoS attacks. The approach try to distinguish three kinds of Users. The malicious user who has fix source IP address and injects spoofed packets to the switch infinitely. The DDoS attacking user sends spoofed packets to the switch infinitely. The frequent user acts as normal user. The method distinguishes them and processes diferently according to diferent users. It works well when the attack trafic is not very huge, and if the attack type it is mainly a flooding one. But to use it, we need to survey the network first and initiate two parameters for the detection method, so it is necessary a kind of "setup" time in order to fix such parameters. The efect of the method may be afected by the artificial parameters. When involved with the behavior of the artificial, the uncertainty of the detecting result will increase. Seyed Mohammad Mousavi et al. [ 6 ] proposed a solution to detect DDoS attacks based on the entropy variation of the destination IP address. Although it is a lightweight and efective detection method, in detecting DDoS attacks, we cannot only take one factor into consideration, since there are many factors that can be used to identify DDoS attacks, which can manifest themselves in very diferent ways beyond the proposed control. The detection method lacks of comprehensive consideration of multi-factors. Uygar Dincalp et al. [ 7 ] proposed a method based on the clustering algorithm DBSCAN for analyzing the network trafic in order to catch the changes and varieties in attack vectors for showing what the attack and where the attack is based on. The proposed methodology strongly depends on a fixed threshold in order to send an alarm of a possible attack, it is not clearly explained how the threshold is defined but the results shows that the proposed system worked well with chosen attributes in their experiments. Akella et al. [ 8 ] proposed a detection mechanism where each intermediate router detects trafic anomalies using profiles of normal trafic. Each router keeps track of destinations whose trafic occupies greater than a fraction of the capacity of the outgoing link, and sends this information to its neighbors. Attack detection is determined by intermediate routers if the gathered trafic information on a specific destination system exceeds the predefined threshold. This scheme cannot distinguish the flash crowds provided by a spike of normal trafic from the DDoS attacks. Hence, false alarm rate will be increased.

2. Dataset

The proposed model is trained and validated on the dataset released by the Canadian Institute for Cybersecurity (CIC), namely CIC-DDoS2019 [ 9 ]. The dataset ofers an extended set of Distributed Denial of Service attacks, most of which employ some form of amplification through reflection. This type of attacks are conducted concealing the attacker's identity thanks to the IP spoofing technique in which packets are sent to reflector servers by attackers with the source IP address set to the target victim's. The dataset contains benign and the most up-to-date common DDoS attacks, this two classes are divided as reported in Figure 1, with a total amount of data of 431,371 records described by 88 features. A slight imbalance is highlighted in the composition of the dataset, where the connections associated with the attacks represent approximately 77% of the total dataset, this slight lack of data for the benign connections would have been possible to fix through oversampling algorithms, creating artificial data based on characteristics of the original ones, the choice in this experiment was not to make changes in this sense, as the imbalance present between the two classes was not too accentuated, equally allowing correct training of the model, the model performance evaluation is validated by applying the cross-validation step.

The following types of attacks are present: UDP, MSSQL, Portmap, Syn, NetBIOS, UDPLag, LDAP, DrDoS_DNS, UDPlag, WebDDoS, TFTP, DrDoS_UDP, DrDoS_SNMP, DrDoS_NetBIOS, DrDoS_LDAP, DrDoS_MSSQL, DrDoS_NTP.

3. Implementation

The proposed work (Figure 2) consists of two main phases: data phase and model phase.

During data phase, the following points are developed: Feature pre-processing, Feature selection, Dimensionality reduction. During the model phase, relating to the development of the Gaussian mixture model, the following points are addressed: Model training, Performance evaluation, Cross-Validation. 3.1. Feature pre-processing The eficiency of classification techniques have to be improved through adequate data manipulation which concerns diferent types of actions. Additionally, models trained on manually prepared data exhibited better performance compared to those trained on non-prepared data according to [ 10, 11, 12 ].

3.1.1. Feature Scaling

Feature scaling is a vital step in pre-processing data before building a model using machine learning [ 13, 14, 15, 16 ]. The dataset used for model training in machine learning often contain unpredictable values that may have varying scales. This can result in inequalities in comparing these values. Feature scaling techniques can address these challenges by adjusting the values and promoting easy and fair comparisons among values.

The ML algorithm observes only numbers, and if there is a significant diference in range, it assumes that numbers in the upper ranges are superior such that features with larger numerical values have a greater efect on the distance between data and dominate other features when calculating distances. As a result, these more significant numbers play a more critical role during model training. The scaling technique used in the experiment is the Normalization, also known as Min-Max scaling, a technique in which values are shifted and rescaled to a range between 0 and 1 without distorting diferences in the ranges of values or losing information.

3.1.2. Encoding

Many statistical learning algorithms require as input a numerical feature matrix, as the case of GMM. When categorical variables are present in the data, feature engineering is needed to encode the diferent categories into a suitable feature vector.

Feature encoding is the process of transforming textual data into numerical values so they may be applied to ML algorithms, resulting in improved model accuracy. Researchers have used many approaches to convert textual data into numerical values, in this work is used the “Label Encoding” technique [ 17 ], in such work this process is applied to the target feature of the dataset in which there are two diferent values, Attack and Benign. Label encoding has the advantage that it is straightforward, yet it has the disadvantage that the numeric values can be “misinterpreted” by the machine learning algorithms since it uses number sequencing. The problem using the number is that they introduce relation/comparison between them. Apparently, there is no relation between Attack and Benign, but when looking at the number, ’Benign’ which is encoded using 1 has higher precedence over ‘Attack’ which is encoded using 0, this kind of problem in this work does not occur.

3.1.3. Feature selection

Many ML models experience dificulty working with a high presence of features in input, generally, features can be categorized as: relevant, irrelevant, or redundant, the last two categories only increase the size of the input space [ 18 ] resulting in dificulty to process data further thus not contributing to the learning process. To generate the best performing model, feature selection plays a major role, which process a subset from available features data are selected for the process of learning algorithm. The best subset is the one with least number of dimensions that most contribute to learning accuracy, since an irrelevant feature does not afect describing the target concept in any way, a redundant feature does not add anything new to describing the target concept. Redundant features might possibly add more noise than useful information in describing the concept of interest. The main benefits of feature selection are follows: (i) reducing the measurement cost and storage requirements, (ii) coping with the degradation of the classification performance due to the finite size of training sample sets, (iii) reducing training and utilization time, (iv) facilitating data visualization and data understanding, (V) reducing the risk of overfitting .

This process can be carried out into three ways [ 19, 20 ]: iflter, wrapper, and embedded.

The feature selection process of the proposed work uses a iflter approach based on the Random Forest Classifier (RFC) as in according with [ 21 ], since it shows the most suitable performance among other filtering approaches, before the application of the model, the dataset is divided into train and test sets in the ratio of 80:20, since feature selection using only the training data (train set) rather than the entire dataset, is particularly important in order to avoid so-called "data leakage". The main reason behind this choice is that, when selecting features, you want the selection to be based only on the information available during the model training phase. If you also use data from the test set during the feature selection phase, you may run the risk of using future information to guide feature selection, introducing a bias into the results. The RFC takes the training dataset and resample it according to a procedure called “bootstrap”. Each sample contains a random subset of the original columns and is used to fit a decision tree. Each tree of the random forest can calculate the importance of a feature according to its ability to increase the pureness of the leaves. The higher the increment in leaves purity, the higher the importance of the feature. This is done for each tree, then is averaged among all the trees and, finally, normalized to 1. So, the sum of the importance scores calculated by a Random Forest is 1. In this paper Information Gain (IG) criteria is used for feature selection by RFC. To use Information Gain for feature selection an entropy value of each attribute of the data has to be calculated. The entropy value is used for ranking features that afect data classification. A feature which does not have much efect on the data classicfiation has very small information gain and it can be ignored without afecting the detection accuracy of a classifier [ 22 ].It calculates the amount of entropy (uncertainty) that is reduced as a result of dividing the data by a specific property. Hence for each splitting attribute, information gain is calculated and the attribute with highest gain is chosen as splitting attribute. This attribute is such that it creates minimum impurity or randomness in the generated splits and hence it minimizes the information needed to classify the tuples. The entropy of a subset S is determined as follows: () = ∑︁( · 2())

=1 Here c = 1, .., C are the diferent classes in S, is probability that an arbitrary tuple in S belongs to class . Let A be a feature in S and 1, 2, . . . , are diferent values of attribute A in S such that 1, . . . , are partitions generated based on these values. These partitions are likely to be impure. How much more information is still needed to arrive at an exact classification or pure partition is given as: () = ∑=︁1( |||| · ()) The smaller is this additional information the greater the purity of the partition.

() = () − () (3)

Once the IG relating to all the features of the dataset has been defined via RFC, the Select From Model class is applied (1) (2) to the results for the selection of the main important features by defining a threshold based on the average of the values coming from the RFC. Then, the features that have the IG above the threshold are selected, which are shown in Figure 3.

3.1.4. Dimensionality reduction

The dataset, after having gone through a pre-processing and feature selection phases, still has a high dimensionality with the presence of 28 features. In this regard, a reduction of the latter is necessary, as we want to have a representation of the data. Dimensionality reduction has been made using Principal Component Analysis (PCA) [ 23 ]. Principal Component Analysis (PCA) simplifies the complexity in highdimensional data while retaining trends and patterns. It does this by transforming the data into fewer dimensions with minimal loss of overall dispersion, which act as summaries of features. PCA reduces data by geometrically projecting them onto lower dimensions called principal components (PCs) defined as a linear combination of the data 's original variables, with the goal of finding the best summary of the data using a limited number of PCs. The first PC is chosen to minimize the total distance between the data and their projection onto the PC. By minimizing this distance, we also maximize the variance of the projected points. The second (and subsequent) PCs are selected similarly, with the additional requirement that they be orthogonal (proving to be uncorrelated) with all previous PCs. Hence, principal components represents the directions of the data that explain a maximal amount of variance, or rather, the lines that capture most information of the data. The relationship between variance and information here, is that, the larger the variance carried by a line, the larger the dispersion of the data points along it, and the larger the dispersion along a line, the more information it has. Hence, PCs are new axes that provide the best angle to see and evaluate the data, so that the diferences between the observations are better visible. Given the dealing of the PCs with distance, a fundamental step required in order to avoid falsification of measurements is data standardization, since heterogeneous data representations going to influence the PCs constructions, given that in case of small set of variables has a much larger magnitude than others, the components in the PCA analysis are heavily weighted along those variables, while other variables are ignored. As a consequence, the PCA simply recovers the values of these high-magnitude variables for this reason standardise the scale of features variation is essential, the consequence that this aspect can have is shown in Figure 4 with a comparison on the first principal component whether features are standardized or not. The ifrst application of PCA shown in Figure 5 is performed by defining a number of 10 PCs to see and analyze how the cumulative variance is distributed over a larger number of components. Subsequently, 3 components were extracted from the results obtained, since with 3 components it is possible to visualize the data, reduce the computational power, ifnding only a data loss of 8%. 3.2. GMM Gaussian Mixture Model (GMM) [ 24 ] is a probabilistic model to describe subsets of data within a general population, that can be represented as a combination of normally distributed subpopulations. It is commonly used for unsupervised learning to learn the subpopulations models that can be also automatically divided. On the other hand within a supervised approach, such models can be used in order to determine the boundaries of diferent subpopulations for classification purposes. In this latter case, the goal is to assign each data point to one of G preexisting unordered classes (or populations) taking into account observed variables, . The problem amounts to define a function that maps an arbitrary observation ∈ to a prediction of the class from which it stems. This function, named allocation rule, has to be estimated from the training data {, = 1, ..., = 1, ..., }, which consist of the observations of vector and class membership for = ∑︀

=1 items. In supervised classification, their use is related to the Bayes allocation rule, which is known to minimize the expected error rate in class membership prediction. The Bayes rule suggests to allocate to the class ˆ having the highest posterior probability among the other classes C, that is: where (|Θ) is the chosen model, is the number of parameters to be estimated and is the number of sample data points. Models with smaller values of BIC are preferable.

Notice as sample size increases, BIC provides an increasingly larger penalty per parameter and thus tends to select ˆ = =1,...,{ ()} = =1,...,{ () (|)} more parsimonious models. In fact, BIC tends to overly (4) penalize complex models, so the goal is to find the model Where denotes the a priori probability of class which for which the result is minimized, preferring the point of are known or estimated from the training data, and () the first "elbow" of the function. denotes the class-conditional probability distribution of .

For continuous predictors mixtures of multivariate normal densities are preferred, because of their computational con- 4. Experimental setup venience: = − 2 log (|Θ) + log , (6) () = ∑︁(ℎ ( | ℎ, Σ ℎ)) ℎ=1

(5) In mixture (5) the components (· ) are d-dimensional Gaussian densities, each parameterized by its mean vector ℎ and covariance matrix Σ ℎ. The parameters ℎ (ℎ = 1, ..., ) are mixing proportions named also mixture coeficient, which are constrained to be positive quantities that sum to 1. These parameters are determined by fitting model (5) to the data {, = 1, ..., = 1, ..., }, usually by maximum likelihood via the expectation-maximization (EM) algorithm [ 25 ].

Then, the model is plugged into rule (4).

3.2.1. Components estimation

In order to improve the performance of GMM as a classifier, we need to define correctly how many Gaussian components should be used to approximate the data distribution, a GMM with too many Gaussian components may overfit the data [ 26 ], while a GMM with too few components may not be flexible enough to approximate the true underlying density distribution, observing an underfitting of the data. The correct components estimation it was done using the Bayesian Information Criterion (BIC) [ 27 ], which is a criterion for grading models based on the posteriori probability of the models being compared. The Bayesian information criterion (BIC) is given by:

4.0.1. Model implementation

In order to have a better prevision on the data, two GMMs are implemented in two diferent ways, one regards only Benign data type while the other one regards only Attack data type. Hence, for extracting the correct number of components of each one of the model, the BIC is applied for each case and the change for diferent component values is presented in Figure 6. Given the obtained results, for what concern the GMM related to Attack data, is chosen ifve as number of components, though corresponding to six components there is the lowest BIC value, performances are equal as using five, so even in terms of computational complexity five was chosen, on the other hand GMM related to Benign data is developed using three components given the clear result. Hence, the two diferent models are defined as follows: () = ∑︁(, ( | ,, Σ ,)) =1 () = ∑︁(, ( | ,, Σ ,)) =1

5. Results

For the classification of a new point, the posterior probability of belonging to the two classes is calculated:

(|) ∈{,} = (|) () , () then, the point is associated with the class ˆ with the maximum posterior probability:

ˆ = ∈{,} { () (|)} The metric used for evaluating the quality of a model’s predictions is the balanced accuracy, which is the mean of Sensitivity and Specificity. Where Sensitivity (True Positive Rate) is the probability of a positive case being accurately classed as being positive, and Specificity (True Negative Rate) is the probability of a negative case being accuracy classed as negative. This specifications helps the metric perform well with the slightly imbalanced dataset used, (7) (8) (9) (10)

Cross-validation performance

Standard deviation 98.7% returning the average accuracy per class. Using the average of Sensitivity and Specificity, we are able to account for imbalanced datasets as a model will receive a worse balanced accuracy score if it only predicts accurately for the majority class in the dataset. As a result, the balanced accuracy report an accuracy equal to 99%. Following is presented a resume of the obtained result using a confusion matrix and then a classification report, over 86275 samples used as a test data:

5.0.1. Cross-Validation

Cross-validation is one of the most widely used data resampling methods to estimate the true prediction error of models. The Cross-validation technique used in the experiment is the Stratified k-fold, in which the available learning set is partitioned into disjoint subsets of approximately equal size, where the presence of data typology is the same as the original dataset. The word “fold” refers to the number of resulting subsets. This partitioning is performed by randomly sampling cases from the learning set without replacement. The model is trained using − 1 subsets, which, together, represent the training set. Then, the model is applied to the remaining subset, which is denoted as the validation set, and the performance is measured. This procedure is repeated until each of the subsets has served as validation set. Consequently, the average of the performances on the validation sets is calculated, which represents the cross-validation performance and subsequently the standard deviation is also calculated considering the result of each validation set compared to the performance score.

6. Conclusions

In this paper, a procedure for fitting Gaussian mixture models (GMM) oriented to supervised classification has been proposed for classifying DDoS attacks. The proposed method demonstrates exceptional adaptability, flexibility, and performance in detecting DDoS attacks, making it a promising solution for enhancing cybersecurity in critical infrastructures. Starting from the raw data taken from the CIC-DDoS2019 dataset, a pre-processing phase was necessary in which the latter were processed with a subsequent selection of the characteristics relevant to the purpose. The results from the test data, provides an accuracy of 99% which is confirmed even by the Cross-Validation with a 98.7% accuracy, highlighting a reliable result without problems like overfitting that in some scenario can conceal the real performance. As part of our future work, first of all, we will be focusing on improving the DDoS attack data, increasing the diversity of the training data to keeps as much types of attack as possible, but even including the normal network trafic data in order to reach a more balanced dataset. This component plays a crucial role in the success of our strategy, as it needs to have a deep understanding of DDoS attacks. To achieve this goal, we plan to automate the process as much as possible. Subsequently, The GMM could be implemented to classify more than two classes, e.g. trying to classify each kind of DDoS attack or trying to model a multi-class problem where the classes could be diferent kinds of cyberattacks.

Acknowledgments

This work has been developed at is.Lab() Intelligent Systems Laboratory at the Department of Computer, Control, and Management Engineering, Sapienza University of Rome (https:// islab.diag.uniroma1.it). The work has also been partially supported from Italian Ministerial grant PRIN 2022 “ISIDE: Intelligent Systems for Infrastructural Diagnosis in smart-concretE”, n. 2022S88WAY - CUP B53D2301318, and by the Age-It: Ageing Well in an ageing society project, task 9.4.1 work package 4 spoke 9, within topic 8 extended partnership 8, under the National Recovery and Resilience Plan (PNRR), Mission 4 Component 2 Investment 1.3—Call for tender No. 1557 of 11/10/2022 of Italian Ministry of University and Research funded by the European Union—NextGenerationEU, CUP B53C22004090006. 1–8

References

[1] N. Hoque , D. K. Bhattacharyya , J. K. Kalita , Botnet in ddos attacks: trends and challenges , IEEE Communications Surveys & Tutorials 17 ( 2015 ) 2242 - 2270 .

[2] S. I. Illari , S. Russo , R. Avanzato , C. Napoli , A cloudoriented architecture for the remote assessment and follow-up of hospitalized patients , volume 2694 , 2020 , pp. 29 - 35 .

[3] Genie-Networks , Ddos attack statistics and trends report for 2020 ( 2021 ).

[4] M. Wang , Y. Lu , J. Qin , A dynamic mlp-based ddos attack detection method using feature selection and feedback , Computers & Security 88 ( 2020 ) 101645 .

[5] N.-N. Dao , J. Park , M. Park , S. Cho , A feasible method to combat against ddos attack in sdn network , in: 2015 International Conference on Information Networking (ICOIN) , IEEE, 2015 , pp. 309 - 311 .

[6] S. M. Mousavi , M. St-Hilaire , Early detection of ddos attacks against sdn controllers , in: 2015 international conference on computing, networking and communications (ICNC) , IEEE, 2015 , pp. 77 - 81 .

[7] U. Dincalp , M. S. Güzel , O. Sevine , E. Bostanci , I. Askerzade , Anomaly based distributed denial of service attack detection and prevention with machine learning , in: 2018 2nd International Symposium on Multidisciplinary Studies and Innovative Technologies (ISMSIT) , IEEE, 2018 , pp. 1 - 4 .

[8] A. Aditya , Detecting ddos attacks on isp networks , in: Proc. of ACM SIGMOD Workshop on Management and Processing of Data Streams , 2003 , 2003 .

[9] I. Sharafaldin , A. H. Lashkari , S. Hakak , A. A. Ghorbani , Developing realistic distributed denial of service (ddos) attack dataset and taxonomy , in: 2019 international carnahan conference on security technology (ICCST) , IEEE, 2019 , pp. 1 - 8 .

[10] B. Xue , M. Zhang, Evolutionary feature manipulation in data mining/big data , ACM SIGEVOlution 10 ( 2017 ) 4 - 11 .

[11] V. Ponzi , S. Russo , V. Bianco , C. Napoli , A. Wajda , Psychoeducative social robots for an healthier lifestyle using artificial intelligence: a case-study , volume 3118 , 2021 , pp. 26 - 33 .

[12] G. Lo Sciuto , S. Russo , C. Napoli , A cloud-based flexible solution for psychometric tests validation, administration and evaluation , volume 2468 , 2019 , pp. 16 - 21 .

[13] G. Capizzi , G. L. Sciuto , C. Napoli , E. Tramontana , A multithread nested neural network architecture to model surface plasmon polaritons propagation , Micromachines 7 ( 2016 ). doi: 10 .3390/mi7070110.

[14] M. M. Ahsan , M. P. Mahmud , P. K. Saha , K. D. Gupta , Z. Siddique , Efect of data scaling methods on machine learning algorithms and model performance, Technologies 9 ( 2021 ) 52 .

[15] G. Capizzi , G. L. Sciuto , C. Napoli , R. Shikler , M. Wozniak, Optimizing the organic solar cell manufacturing process by means of afm measurements and neural networks , Energies 11 ( 2018 ). doi: 10 .3390/ en11051221.

[16] S. Pepe , S. Tedeschi , N. Brandizzi , S. Russo , L. Iocchi , C. Napoli , Human attention assessment using a machine learning approach with gan-based data augmentation technique trained using a custom dataset , OBM Neurobiology 6 ( 2022 ). doi: 10 .21926/ obm.neurobiol. 2204139 .

[17] F. Pargent , F. Pfisterer , J. Thomas, B. Bischl , Regularized target encoding outperforms traditional methods in supervised machine learning with high cardinality features , Computational Statistics 37 ( 2022 ) 2671 - 2692 .

[18] B. Bhattacharya , D. P. Solomatine , Machine learning in sedimentation modelling , Neural Networks 19 ( 2006 ) 208 - 214 .

[19] A. Jović , K. Brkić , N. Bogunović , A review of feature selection methods with applications, in: 2015 38th international convention on information and communication technology, electronics and microelectronics (MIPRO) , Ieee, 2015 , pp. 1200 - 1205 .

[20] V. Ponzi , S. Russo , A. Wajda , R. Brociek , C. Napoli , Analysis pre and post covid-19 pandemic rorschach test data of using em algorithms and gmm models , volume 3360 , 2022 , pp. 55 - 63 .

[21] M. A. Salam , A. T. Azar , M. S. Elgendy , K. M. Fouad , The efect of diferent dimensionality reduction techniques on machine learning overfitting problem , Int. J. Adv. Comput. Sci. Appl 12 ( 2021 ) 641 - 655 .

[22] B. Azhagusundari , A. S. Thanamani , et al., Feature selection based on information gain , International Journal of Innovative Technology and Exploring Engineering (IJITEE) 2 ( 2013 ) 18 - 21 .

[23] F. L. Gewers , G. R. Ferreira , H. F. D. Arruda , F. N. Silva , C. H. Comin , D. R. Amancio , L. d. F. Costa , Principal component analysis: A natural approach to data exploration, ACM Computing Surveys (CSUR) 54 ( 2021 ) 1 - 34 .

[24] D. A. Reynolds , et al., Gaussian mixture models ., Encyclopedia of biometrics 741 ( 2009 ).

[25] B. Barazandeh , M. Razaviyayn , On the behavior of the expectation-maximization algorithm for mixture models , in: 2018 IEEE Global Conference on Signal and Information Processing (GlobalSIP) , IEEE, 2018 , pp. 61 - 65 .

[26] V. Nannen , The paradox of overfitting , Ph.D. thesis, Faculty of Science and Engineering , 2003 .

[27] G. Schwarz , Estimating the dimension of a model , The annals of statistics ( 1978 ) 461 - 464 .