The article contains a detailed data analysis based on the Email Spam Classification database. The results of using various methods of column elimination as well as classification and regression algorithms will be presented.
Almost every area of life has been computerized. At
every step we are dealing with more and more complex
IT systems, which thanks to equipping with artificial
intelligence systems [
Did you ever wonder how often people check their mailboxes? According to the USPS, 98% of people visit their mailbox every day to get their mail and 77% of people sort through their mail immediately after they wake up. Nowadays, the mailbox is a big part of our daily lifestyle. There are a lot of programs for mailbox from where we can choose the one that suits us best, for example Apple Mail, Mail App for Gmail, Outlook, Spark, and BlueMail. There is one major function that all of those software includes – all of them can detect the infuriating messages from shops, organizations, or websites where you had logged in only once to get the coupon for 20% of the electric toothbrush. We all perfectly know how annoying are those emails called spam. But have you ever taken into consideration how the email spam detector in your daily mailbox works? We were interested in investigating how does the tool we use and love everyday works this much that we took our research to prepare and to comapre the best data analysis methods for email spam detector using KNN and Naïve Bayes Classifier.
The main task in our project is to detect spam among the emails. At first program in purpose to divide spam and non-spam messages are based on the analysis of the frequency of occurrence of given word in their content. For the purposes of the project, the program first reduces the size of the database. Then it eliminates the columns, leaving the significant ones, with a few diferent algorithms. Finally with the Naive Bayes algorithm and KNN we are detecting witch of the emails are called a spam.
The database on which we conduct our tests is the email spam classification database. It contains 3002 columns and 5172 lines, where each line represents a separate mail. The columns show the most common words in all e-mails. The last column contains the predictive labels: 1 for spam and 0 for no spam. The name was set with numbers, not the names of the recipients, to protect privacy. Information is stored in a compact data frame. The base was taken from the website available in this link [24] where you can also learn more about it.
Initially, the complete database is divided into spam and non-spam. Then selected are randomly 1500 records from two those separate groups. The data is then combined successively and mixed, and then the newly created database will be subject to further analysis.
There are many ways to select only useful features in database, but those methods for feature selection can be divided into three main groups:
This is another filtering method. The chi-square test is used to determine if there is a significant diference between the expected and observed frequencies in one or more categories. Then it considers if there is a sampling error.
In this method, we calculate the chi-square metric between the target variable and the numeric variable. The algorithm then selects only the variables with the maximum chi-square values. To correctly compute chisquare, we first find the values that we would expect in each part if there were indeed independence between the two categorical variables.
Finally, follow the formula below; we multiply the sum of the rows and the sum of the columns for each cell and divide it by the sum of the observations.
2 = ∑︁ ( − )2 (2)
Due to the size of the base and the number of columns, there was a need to eliminate them. Therefore, 6 methods will be used for this purpose. However, first it is worth presenting the types of these methods.
• Filter based: we define some data and metric and This is a wrapper method. Wrapping methods treat based on these filter functions. An example of the selection of a feature set as a search problem. such a metric may be a chi-square. When using RFE, there are two important configura• Wrapper-based: Those methods consider the se- tion options: choosing the number of functions to select lection of a set of features as a search problem. and selecting the algorithm used to select the functions. An example of such methods may be a Recursive Both of these hyperparameters can be tested, although Feature Elimination. the method’s performance is not strongly dependent on • Embedded : Embedded methods use algorithms the correct configuration of these hyperparameters. which have built-in feature selection methods. From the sklearn documentation: For instance, Lasso. “The goal of recursive feature elimination (RFE) is to select features by recursively considering smaller and smaller sets of features. First, the estimator is trained on an initial feature set and the importance of each feature is obtained either through the coef attribute or through the feature-importances attribute. Then, the least important features are removed from the current feature set. This procedure is repeated recursively on the truncated set until the desired number of features to be selected is ifnally reached.”
This is one of the filtering methods. First, we check the absolute value of the Pearson correlation between the target and the numerical features in the dataset being checked. Based on this criterion, we keep the n best functions. The Pearson correlation coeficient r is used to test whether two quantitative variables are related to 6.4. Lasso
This is an embedded method. As mentioned earlier, embedded methods use algorithms that have built-in function selection methods. For example, Lasso and RF have function selection methods. Lasso sets parameters to zero. The higher the alpha value, the fewer features have non-zero values.
∑︁ ||
=1
It is also an embedded method. Team machine learning for classification, regression, and other tasks, which involves building multiple decision trees while training, and generating a class that dominates the classes or the predicted mean of each tree. We can also use RandomForest to select functions based on their importance.
We compute the importance of the features using the nodal impurities in each decision tree. In a random forest, the final importance of a feature is the average importance of all the features of the decision tree.
PCA is a linear dimension reduction technique that can be used to extract information from a high dimension space by projecting it onto a lower dimension subspace. The algorithm tries to keep the most important parts while removing the less important ones.
One important thing to note with PCA is that it is an unsupervised dimension reduction technique where you can group similar data points based on the correlation of features between them without any supervision. PCA is a statistical procedure that uses an orthogonal transformation to transform a set of observations of potentially correlated variables into a set of values for linearly uncorrelated variables, called principal components. The way it works is that if the correlation among a subset of features is definitely high, the algorithm tries to combine similar features.
Thanks to these components, it is possible to recreate the original characteristics suficiently accurately. The PCA algorithm actively tries to minimize the reconstruction error when searching for optimal components.
Comparison of column elimination algorithms for different number of columns using the Bayes algorithm.
When analyzing the table above, it can be seen that good results of 80% can be obtained in some algorithms with as little as 10 columns.
The 20 columns were adopted for further consideration to obtain better results and maintain eficiency.
Sample correlation matrix for Pearson algorithm and 20 columns. 16–23 (4)
(3)
A Naive Bayes classifier is a probabilistic machine learning model that’s used for classification task. The crux of the classifier is based on the Bayes theorem. (|) = (|) () ()
Naive Bayesian classifiers are based on the assumption that predictors (independent variables) are independent of each other. They often have nothing to do with reality because of their contractual assumptions and are therefore called naive.
These algorithms are most often used in sentiment analysis, spam filtering, and recommendation systems. They are quick and easy to implement, but unfortunately have the downside of requiring the independence of the predictors.
The k-nearest neighbor algorithm, also known as KNN, is a non-parametric supervised learning classifier that uses proximity to create a classification or group prediction for a single data point. It is used for regression and classification, but more often for the second application, which is based on the assumption of proximity of points.
The K Nearest Neighbors method belongs to the group of lazy algorithms. This is because it does not create an internal representation of the training data, but only looks for a solution when a test pattern appears. It stores all the training patterns on the basis of which it determines the distance of the test pattern.
Learning an algorithm can be represented in three ways. The first is instance-based, which uses whole training instances to predict the result. Another is Lazy Learning, which is distinguished by the fact that the model training process is postponed until the program requests a forecast in a new instance. The last one is non-parametric, meaning there is no predefined form of the mapping function.
The data set was divided into a training and validation set on a scale of 70:30.
Test results for all column elimination algorithms and Bayes’ algorithm for non-normalized data.
The best result was obtained using the Ch-Square algorithm and the worst for the Lasso algorithm. It is worth adding that all the results do not difer too much from each other.
Additionally, tests were carried out for the combined above five algorithms. Below you can see the statistics and results of word matches during the test. As it turns out, we managed to get the correctness at the level of exactly 80%, which is less than in each algorithm separately.
The results of the tests performed for all column elimination algorithms along with the combined version for the Bayes algorithm for normalized data.
As can be seen, normalization does not have a significant impact on the results as the data is very comparable. However, in this case, the highest accuracy was calculated for the Pearson algorithm and the worst for Recursive Feature Elimination and Lasso.
The PCA algorithm for two and three components
As part of the experiment, the KNN algorithm was was also tested. As can be seen, the results for three are tested for a diferent number of columns, as shown in definitely better than for two. the first table, and for diferent elimination algorithms, It is also worth comparing this algorithm with those as shown in the second table, for k equal to 3, 5, 7, 9, tested earlier. It can be concluded that the best result of respectively. all is obtained by using the PCA algorithm for the three
The first table shows that the best results are obtained components. for 20 columns, and with more and fewer of them, the effectiveness decrease. On the other hand, from the second table, we learn that it is best to use the Pearson algorithm.
Based on these conclusions, further tests were performed for diferent k for non-normalized and normalized data.
Numbers of features
Accuracy
particle filter for mobile robot localization based on particle swarm optimization, Expert Systems with