In the digital world, one of the most significant threats is malicious software, known as malware, developed by cyber attackers to intentionally cause damage to computer systems or gain access to them. The form and behavior of malware have developed year by year at the same time methods of detection from malicious software have evolved to ensure security. Earlier for disclosure of malicious software some classical methods, such as signature-based, heuristic, and so on. Traditional methods for detecting malware have failed to defeat new generations of malware and their sophisticated obfuscation tactics. However, at present, the use of detection methods based on machine learning has been recognized as one of the most modern and prominent methods. Methods based on machine learning provide fast malware prediction with excellent detection and analysis rates for various types of malware. Therefore, this research work represents a comparison of various machine learning approaches, including neural networks, decision trees, the support vector machine, and ensemble methods applied to the analysis of behavioral and static characteristics of malware. The research paper analyzes the advantages and limitations of each method, their effectiveness in various types of malicious attacks, and the possibilities of their adaptation to changing threats. The current trends and directions of development in the use of machine learning in malware forensics, including the use of deep learning and big data analysis technologies, are also considered. The objective of this research is to conduct a comparative analysis of various machine learning techniques applied in malware forensics, focusing on their effectiveness in detecting, classifying, and analyzing malicious software. The study aims to identify the strengths and weaknesses of these techniques in real-world digital forensic scenarios, providing insights into their applicability and performance in enhancing malware investigation processes. The results of the study emphasize the prospects of using machine learning to improve malware detection and control, as well as the importance of further research to develop new models and methods of data analysis to protect information systems from modern cyber threats.
Malicious software, known as malware, is developed to harm digital devices on purpose. Nowadays, there are many types of malware, such as trojan horses, worms, viruses, bots and botnets, ransomware, adware, and spyware. Attackers targeted individual computers and networks, which led to an increase of holes in the system of security. The most significant issue in this situation is confidentiality and leakage of personal information. Moreover, financial damage caused by malware software all over the world increased proportionally. According to the report of McAfee about COVID-19 Threats and Malware Surges, the new malware samples averaged 648 new threats per minute [1]. Also, recent estimates of the AV-Test show that every day over 450,000 new malicious programs and potentially unwanted applications [2]. To ensure an answer to cyber threats, which are complex and unsafe, most of the farms developed different instruments for cyber security. Researchers used machine learning and deep learning algorithms to create effective models to solve these problems by conducting detailed research. Malware protection methods are also becoming more sophisticated as the variety of malware increases.
Earlier, malware was written by using simple codes and they were easily detected. Currently, creators of malicious software complicate the code, as a result even advanced methods cannot detect them. The next generation of malicious programs is hard to identify by comparison with traditional malicious programs designed to run in the kernel. These types of malware programs slip away easily 0000-0003-3913-1895 (B. Abduraimova); 0000-0003-4992-0564 (S. Gnatyuk); 0009-0009-7123-9522 (A. Nurmukhanbetova) © 2024 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). from the security system, such as firewalls and antiviruses. There are different approaches to malware detection, based on different functions, such as signature, heuristic, behavior, model verification, cloud, mobile devices, Internet of things, machine learning, and deep learning. At first, the detection of malware programs used signature-based detection. Despite its speed, it cannot identify complex malware. Traditional signature-based methods, such as pattern matching, do not meet the requirements for malware detection. So, in this case, the use of machine learning algorithms is suitable for the detection of malicious software now [3].
This paper represents malware detection using machine
learning algorithms and recommends each of them. In
Sections 1 and 2 we review traditional and machine learning
methods of detection of malicious software. Section 3
contains a detailed comparison table of machine learning
algorithms and shows the practical part in Python. Finally,
section 4 keeps the conclusion and final notes [
Signature-based detection: In this approach, malware is
detected based on certain signatures or file patterns. This
traditional method allows the detection fastly known
malicious software compared to other methods. Most of the
antivirus programs are realized by using signature-based
methods. Signature of malware generated and stored in the
database of detection to verify the signature of an unknown
file. If the signature of the file matches, then it is declared a
malicious file, otherwise, it is a benign file. The main issue
of this method is that the file signature changes even when
one byte of the file is changed. Thus, for every modified and
new malware must be generated a new signature. After that,
the detector of malicious software can find a new signature.
Also, the main disadvantage of this detection is that it takes
a lot of time and effort to detect malware and extract
different signatures for different types of malware [
Behavior-based malware detection: This method is
based on program behavior monitoring and decision
making whether it is malware or not. This method detected
malicious software by its suspicious behavior difference
from other typical programs. The behavior of malware
determines its importance in this type, and sometimes
different malware programs are found under the same
signature [
Heuristic-based malware detection: This method studies
the behavior of malware by using machine learning
algorithms and intellectual data analysis. It solves many
problems, which are existing in signature-based and
behavior-based detection methods [
Machine learning is modern technology, which is supposed
to be learning machines based on experience. Machine
learning can detect hidden patterns, which users can detect
easily themselves [
Secondly, an unsupervised model uses data sets without answers. Here, the algorithm classified input data based on the similarity between these values.
Analysis of malicious software includes the process of
analyzing and understanding the malicious software for
determination of its functionality, behavior, and potential
impacts. Analysis of malware provides a clear view of
malware in the code and byte string values, which is used
by an attacker to steal information or modify the source
code. In the process of analysis, various key features of the
code are extracted, which provides information and
functionality of malware activity in the system or network
system [
Static analysis: Static analysis of malicious software includes the study of malware or files without execution.
This method of analysis allows users and analytics of
security to understand structure, behavior, and potential
threats, coming from malware, without the risk of
infecting the system.
Static analysis can present valuable information about
behavior, opportunities, and potential influence of
malware on systems. This analysis is useful for the fast
detection of known malware signatures and the
extraction of important information from the code or
files [
Dynamic analysis: At the same time, malicious code
is executed in a controlled environment to observe its
behavior and understand its capabilities. Difference
from the static analysis that researches the code without
its execution [
Observing malicious software in action, analysts can
gain insight into its purpose, distribution methods, and
the potential damage it can cause [
Hybrid malware analysis: Hybrid malware analysis,
also known as combined analysis or integrated analysis,
is an approach combining several methods, such as static
analysis, dynamic analysis, and behavior analysis, to get
a complete understanding of malware. Using the strong
sides of different analysis methods, hybrid analysis is
aimed at overcoming the limitations of individual
methods and providing a more reliable and accurate
assessment of the behavior, capabilities, and potential
impact of malware [
Machine learning is the subset of AI, used for tasks of detection and classification in different areas.
Algorithms of machine learning, such as RF, NB, KNN, SVM, and DT, are widely used for the detection and classification of malicious programs.
Naive Bayes (NB) is an algorithm of classification based on supervised learning. It works in the base of probability functions, in which each attribute belongs to a specific class. A strong assumption is required that the attributes are conditionally independent. An assumption, which is used in algorithm NB simplifies the probability calculation. It calculates whether a data point can fall into a certain class or not. It can accurately predict test data sets for binary and many class datasets. It is effective and scales well, therefore, it works well for small irrelevant data sets. It used the NB theorem for the decomposition of conditional probability. However, NB gives poor performance because attributes are highly correlated with each other, and it is believed that attributes should be independent.
K-Nearest Neighbors is a nonparametric supervised algorithm, which means that it doesn't make any assumptions about the underlying distribution of the data. It works based on the proximity of the similarity of objects or searching for Nearest Neighbors in the specified set and uses the majority of votes to classify a new data point. KNN defines the similar characteristics of new points on the base of previously saved data points, using the Euclidean distance between two data points. This algorithm is known as lazy learning because it does not require parameter adjustment and works without training the model. Instead of training the model, it takes all the data points at the time of forecasting. However, it has some disadvantages, such as high cost, low speed, and less scalability for large data sets.
Decision Tree is a classification of supervised learning, used for tasks of classification and regression.
It can also work with numbers and category data. It
follows the tree model approach, in which the data
points are divided into two branches and a conclusion
about existing features in each node of the tree [
Pros Cons Fast and safe. High accuracy. It is impossible to analyze confusing and encrypting Low memory consumption. malware. Unknown malware cannot be detected. Can analyze confusing and encrypting Consumes a high number of resources and is unsafe malware. Also, it can detect known and and slow. unknown malware.
The result is more accurate than static Higher complexity and more time are used. and dynamic analysis.
A decision tree is designed for the creation of a learning
flowchart structure, which can be used for the
classification of classes or values targeted variables on
the base of decision-making rules, which are extracted
from previous data. It can work well with huge and
noisy data sets compared to models of KNN and SVM
[
In the data sets of SVM a set of data points separated by
a line called a hyperplane is specified, which is used for
classification of data sets classes. This is an effective
classifier for effectively processing a large set of data. It
is used for problem decisions of linear functions in
multidimensional feature spaces. However, this does not
work well in the case of big data sets with noisy data sets
and requires more time for learning [
Random Forest is a classifier based on ensemble
learning, which consists of several decision trees, that
work parallel, a forest that is constructed and is called
decision tree ensemble of the decision tree. Each tree in
the ensemble model contains data samples taken from
the training data with replacement, known as a
bootstrap sample, which is a set of learning models to
improve the overall results of the models [
It works for categorical and continuous data. However,
the speed of Random Forest is low, because it requires
more time on learning a set of decision trees for
constructing strong classificatory [
Confusion matrix is a matrix that is constructed for every classification model prediction and it shows the number of test cases correctly and incorrectly classified. It looks like this, which is shown in Table 3 (considering 1 (positive) and 0 (negative) are the target classes):
+ + + + It can implement many general issues, but it is not useful when we are talking about unbalanced data sets. For example, if we detect fraud in bank datasets, the ratio of fraud cases to non-fraud cases can be 1:99. In this case, if we use the accuracy, the model can prove to be 99% accurate, predicting all test cases as non-fraud. This way, accuracy is a false indicator of the model's performance, and in this case, the metric is required that can focus on fraud data points.
First of all, for doing research, necessary libraries, such as scipy, seaborn, and Tensorflow to analyze the data, build graphs, and study neural networks to find and classify images, which is shown in Fig.5.
After that, this code which is shown below imports pandas, numpy, tensorflow, keras, and matplotlib libraries for working with data, creating and training machine learning models, and visualization. Fixed random number generator seeds are set to ensure reproducibility of results. Some tools for data preprocessing and model evaluation are commented out and are not currently used.
This code loads data from a CSV file using pandas and stores it in the variable data. It then prints the total number of missing values in this dataset, using the isna() method to determine the missing values and the sum() function to count them. Finally, the code prints the DataFrame data itself. This code parses the unique values and their counts in the “class” column, performs label encoding using “LabelEncoder”, replaces special characters with “NaN”, removes rows with missing values, converts all columns to numeric data types, and outputs the total number of features in the DataFrame after all transformations. The code, which is shown in Fig.8 prints the total number of features in the DataFrame (excluding the class column), and then creates and displays a bar chart of the class distribution, showing the number of instanes for each class. oversamples the minority class to equalize the number of instances with the majority class, and then merges and shuffles the balanced dataset.
The code below balances the classes in a dataset by counting the number of instances of each class and outputting that data. It then splits the data into training and testing sets, where 80% is used to train the model and 20% is used for testing, and outputs the sizes of both sets.
Below is the code that builds a heat map of correlations between features in a dataset. It uses the seaborn and matplotlib libraries for visualization, displaying correlation coefficients as a color map. The heat map helps to identify dependencies and relationships between different features.
The code below trains a decision tree classifier on
the training data, makes predictions on the test data, and
computes the accuracy of the model.
The code, which is shown in Fig. 12 creates a graph
showing the accuracy of different classification
algorithms. It plots a bar chart using a color map to show
the accuracy and adds labels to each bar. It also displays
a color scale indicating the accuracy range.
This paper represented malware detection methods
using machine learning along with basic concepts such
as malware detection programs, and machine learning.
Also, there were several classification algorithms and
the accuracy of methods varied depending on the
method used, the number of attributes, data sets,
preprocessing methods, as well as tools implemented in
the model. It also depends on the analysis method and
the function used. As seen in Table 4, the best possible
malware detection accuracy is a Sequential Neural
Network. This algorithm achieves about 100% [
Sequential Neural Network refers to one of the neural network’s architecture types, where data is processed sequentially, step by step. This is useful for tasks, including time series data, such as text or speech. Based on work, this algorithm reaches the best results. However, it should be recognized that this conclusion does not allow us to claim that the Sequential NN is the best algorithm for detecting malware. There are always other measures to take care of, such as the dataset, preprocessing methods, extracted features, and feature selection methods used.
Further research directions include several key aspects. First, it is worth exploring the possibilities of developing hybrid models that combine different machine learning algorithms to improve overall efficiency and robustness to changing data conditions. Second, it is worth focusing on improving the interpretability of complex models such as neural networks to make them more suitable for forensic practice. Third, it is necessary to explore adaptive learning methods that will allow models to quickly respond to the emergence of new types of malware. It is also worth conducting a comparative analysis of algorithms in different digital forensics scenarios to identify specific requirements and optimal solutions for each task. Finally, it is important to investigate the impact of different tools and frameworks for implementing machine learning on the analysis results, which can significantly improve the practical application of these technologies in forensic science.