As machine-learning (ML) based systems for malware detection become more prevalent, it becomes necessary to quantify the benefits compared to the more traditional anti-virus (AV) systems widely used today. It is not practical to build an agreed upon test set to benchmark malware detection systems on pure classification performance. Instead we tackle the problem by creating a new testing methodology, where we evaluate the change in performance on a set of known benign & malicious files as adversarial modifications are performed. The change in performance combined with the evasion techniques then quantifies a system's robustness against that approach. Through these experiments we are able to show in a quantifiable way how purely ML based systems can be more robust than AV products at detecting malware that attempts evasion through modification, but may be slower to adapt in the face of significantly novel attacks.
The threat and impact of malicious software (malware) has continued to grow every year. The annual financial impact is already measured in the hundreds of billions of dollars (Hyman 2013; Anderson et al. 2013). Simultaneously, there are worries that the classical anti-virus approach may not continue to scale and fail to recognize new threats (Spafford 2014).
Anti-virus systems were historically built around signature
based approaches. Wressnegger et al. (
Given these issues and urgency, Machine Learning would appear to be a potential solution to the problem of detecting new malware. Malware detection can be directly phrased as a classification problem. Given a binary, we define some feature set, and learn a binary classifier that outputs either benign or malicious. The output of this model could be calibrated to reach any desired false-positive ratio.
However, one should never switch to a new technology for its own sake. It is necessary to have empirical and quantifiable reasons to adopt a new approach for malware detection. Ideally, this would be based off the accuracy of current anti-virus systems compared to their machine-learning counterparts. In reality, this is non-trivial to estimate, and many arguments currently exist as to how this should be done (Jordaney et al. 2016; Deo et al. 2016; Sommer and Paxson 2010). Designing a corpus and test protocol to determine accuracy at large is hampered by issues like concept drift (Jordaney et al. 2017), label uncertainty, cost (Miller et al. 2016), and correlations over time (Kantchelian et al. 2013; Singh, Walenstein, and Lakhotia 2012).
Toward remedying this issue, we propose a new testing methodology that can highlight a detector’s strength or weakness with regard to specific attack types. We first utilize the framework from Biggio et al., to specify a model for our adversary (Biggio, Fumera, and Roli 2014). After defining the adversary’s goal, knowledge, and capabilities, we look at how difficult it is for an adversary with known malware to evade malware detection systems under comparison. As examples, we compare two machine learning approaches, one a simpler n-gram model and one neural network based, to a quartet of production AV systems. While we do not exhaustively test all possible evasion techniques, we show the potential for our protocol using a handful of relevant tests: non-destructive automated binary modification, destructive adversarial attacks, and malicious injection. We are specifically looking only at black box attacks which can be applied to any potential malware detector, assuming that the adversary is looking for maximal dispersion and not a targeted attack. This new evaluation protocol is our primary contribution, and a black-box adversarial attack for byte-based malware detectors our second contribution. This evaluation protocol allows us to make new conclusions on the potential benefits and weakness of two byte based machine learning malware detectors.
We will review previous work in section 2. The experiments and methodology we use to compare our ML based systems to anti-virus will be given in section 3 and section 4, followed by their results in section 5 and conclude in section 6.
2
Malware detection and analysis has been an area of active research for several years. One class of techniques that is of particular note is dynamic analysis, where the binary itself is run to observe its behavior. Intuitively, malicious behavior is the best indicator of a malicious binary — making dynamic analysis a popular approach. However, dynamic analysis has many complications. It requires significant effort and infrastructure to make it accurately reflect user environments, which are often not met in practice (Rossow et al. 2012). Furthermore, the malware author can use a number of techniques to detect dynamic execution, hide behavior, or otherwise obfuscate such analysis. This means effective dynamic analysis systems are often a cat-and-mouse game of technical issues, and can require running binaries across many layers of emulation (Egele et al. 2017).
It is for these reasons that we focus on the static analysis case. This removes the unlimited degrees of freedom provided by actually running code. However, this does not mean the malicious author has no recourse. Below we will review the related work in this area.
Questions regarding the security of machine learning based systems have been an active area of research for over a decade (Barreno et al. 2010), but have recently received increased attention. This is in particular due to the success in generating adversarial inputs to neural networks, which are inputs that induce an incorrect classification despite only minor changes to the input (Szegedy et al. 2014). These approaches generally work by taking the gradient of the network with respect to the input, and adjusting the input until the network’s output is altered. This does not directly translate to malware detection when using the raw bytes, as bytes are discontinuous in nature. That is to say, any change in a single byte’s value is an equally “large” change, whereas in images, adjusting a pixel value can result in visually imperceptible changes. Arbitrary byte changes may also result in a binary that does not execute, making it more difficult to apply these attacks in this space (Grosse et al. 2016; Russu et al. 2016). While such attacks on individual models are possible (Kolosnjaji et al. 2018; Kreuk et al. 2018), we are concerned with attacks that can be applied to any detector, therefore these methods are out of the scope of this work.
To overcome the issue with arbitrary byte changes breaking
executables, Anderson, Filar, and Roth (
Another approach is to modify the bytes of a binary, and
run the malware detection system after modification. While
this risks breaking the binary, it can still be effective, and it
is important to still quarantine malware even if it does not
execute as intended. Wressnegger et al. (
For the machine learning based malware detection
methods, we look at two approaches: byte n-grams and neural
networks. The byte n-gram approach was one of the first
machine learning methods proposed for malware detection
(Schultz et al. 2001), and has received significant attention
(Kolter and Maloof 2006). We use the same approach for
training such models as presented in Raff et al. (
Using these modeling methods is beneficial for our analysis because they are trained with minimal assumptions about the underlying data. Other approaches to malware detection that process the PE-header (Shafiq et al. 2009) or use disassembly (Moskovitch et al. 2008) exist. We avoid these under the belief that their use of explicit feature sets aids in evading them (e.g, modifying the PE-Header is easy and allows for direct evasion of models using those features).
We leverage the existing framework from (Biggio, Fumera, and Roli 2014) to model our adversary for each comparison made. This framework prescribes defining the adversary’s goals, knowledge, and capabilities in regards to the classifiers. 3
For each experiment we specify the model of our adversary by outlining the goals, knowledge, and capabilities available (Biggio, Fumera, and Roli 2014). These make clear the assumptions and use cases presented by each scenario.
The goal of our adversary is the same across all experiments – using a single attack to maximize the misclassification rate of malicious files across many classifiers. The one caveat is that in subsection 4.2 we only goes as far as identifying a small portion of the file which can be altered to evade detection. A real attacker would then have to alter those bytes while retaining the malicious functionality of the file. It is unlikely that authors of benign software would seek a malicious classification – therefore we limit ourselves to experiments on malware only.
We severely constrain the knowledge of our adversary. In all cases we assume the adversary has no knowledge of the classifiers’ algorithms, parameters, training data, features, or decision boundaries. This is the best case scenario for security applications, as knowledge of any of the previous attributes would increase the sophistication of possible attacks.
Similarly, the experiments outlined in subsection 4.1 and subsection 4.3 require no prior interaction with the classifiers before deploying a malicious file to the wild. In subsection 4.2, the adversary has the capability of querying our n-gram model only. The inner workings of the model remain hidden, but the output of the model is available for deductive reasoning.
These assumptions are restrictive to the adversary’s actions and knowledge, but are important because current adversaries are able to succeed under such restrictions today. This means malware authors can obtain success with minimal effort or expense. Through our evaluation procedure we are able to better understand which techniques / products are most robust to this restricted scenario, and thus will increase the effort that must be expended by malware authors.
We now perform experiments under our proposed testing methodology to compare machine learning models to commercial anti-virus systems. To do so, we compare two machine learning classifiers and four commercial anti-virus products: AV1, AV2, AV3, and AV4. We use these anonymous names to be compliant with their End User License Agreements (EULA), which bars us from naming the products in any comparative publication. We are prohibited from using the internet based mass scanners, like VirusTotal, for similar reasons. Ideally, we could test many products, but with our large corpus of files we limit ourselves to these four – which we believe to be representative of the spectrum of commercially used AV platforms. We purposely did not chose any security products which advertise themselves as being primarily powered by machine learning or artificial intelligence. Additionally, any cloud based features which could upload our files were disabled to protect the proprietary nature of our dataset. Our machine learning based approaches will be built upon n-grams and neural networks which process a file’s raw bytes. We will compare these systems on a number of tasks to evaluate their robustness to evasion.
We now describe our new testing protocol. Given a corpus fx1; : : : ; xng of n files with known labels yi 2 fBenign; Maliciousg, and detector C( ) we first measure the accuracy of C(xi); 8i. This gives us our baseline performance. We then use a suite of evasion techniques 1( ); : : : ; K ( ) and look at the difference in accuracy between C(xi) = yi and C( (xi)) = yi. The difference in these scores tells us the ability of the detector C( ) to generalize past its known data and catch evasive modifications. Any approach for which C(xi) 6= C( j (xi)) for many different evasion methods which preserve the label yi, is then an intrinsically brittle detector, even if it obtains perfect accuracy before is used.
The raw detection accuracy is not the point of this test. Each system will have been built using different and distinct corpora of benign and malicious files. For the AV products, these are used to ensure that a certain target level of false positives and false negatives are met. In addition, the training corpus we use for our models is orders of magnitude smaller than what the AV companies will have access to. For this reason we would not necessarily expect our approaches to have better accuracy. The goal is to quantify the robustness of any classifier to a specific threat model and attack.
One novel and three pre-existing techniques (“ ”s) will be used to perform this evaluation. These are not exhaustive, but show how our approach can be used to quantify relative performance differences. Below we will review their methodology, and the information that we can glean from their results. In section 5 we will review the results of running these experiments.
We take only a brief moment to expound upon the machine learning models and data used for training, as the information disparity with other AV products makes direct comparison difficult. We trained the machine learning models on a large dataset from industry discussed in (Raff and Nicholas 2017) which contains approximately 2 million samples balanced almost evenly between benign and malicious. Our test set consists of approximately 80,000 files held out for our posttraining experimentation. The test set is also almost perfectly balanced. This set was not seen by our machine learning models during training, but could have files previously seen by the anti-virus corporations. The n-gram model follows the procedure used in (Raff et al. 2016), but uses one million features instead of 100,000. The MalConv approach is specified in (Raff et al. 2017). Both of these approaches require no domain knowledge to apply. 4.1
In this experiment, we use a subset1 of the modifications used
by Anderson, Filar, and Roth (
There are nine different modification actions, and grouped by type are: • rename or create new sections • append bytes to the end of a section or the file • add an unused function to the import table • create a new entry point (which jumps to the old entry) • modify the header checksum, the signature, or debug info
Each malicious file was scanned before the experiment to see if it was already being misclassified as benign (false negative). If it was correctly classified as malicious then a modification was randomly selected from the set and applied. This process was repeated up to ten times or until the file evaded the classifier. Approaches that rely too heavily on exact signatures or patterns should suffer in this test, and we
would expect any approach that uses purely dynamic features to be unaffected by the modifications.
This experiment is time intensive, and so we use a subset of 1,000 files randomly chosen from our test set. Anderson, Filar, and Roth limited themselves to a small 200 file sample, so our larger test set may provide additional confidence in the results. The original paper proposed using a reinforcement learning algorithm (RLA) to select the modifications in an adversarial manner. We found that including a RLA did not change results, but did limit our ability to test with all products in a timely manner. 4.2
The first approach described requires extensive domain knowledge based tooling, and would be difficult to adapt to new file types. We introduce a novel approach to find important byte regions of a malicious file given a working detector. Identifying the most important region of a file gives feedback to adversaries, and allows us to occlude that region as an evasion technique.
Our approach works by occluding certain bytes of a file, and then running the malware detector on the occluded file. By finding a contiguous region of bytes that reduced the detector’s confidence that a file is malicious, we can infer that the bytes in that region are important to the detector. If occluding a small region causes the detector to change its vote from malicious to benign, then we can infer that the detector is too fragile. In this case it is plausible for a malware author to determine what is stored in the small region detected, and then modify the binary to evade detection.
Manually editing groups of bytes one at a time would be computationally intractable. Given a file F of jF j bytes, and a desired contiguous region size of , it would take O(jF j) calls to the detector to find the most important region. We instead develop a binary-search style procedure, which is outlined in Algorithm 1. Here C( ) returns the malware detector’s confidence that a given file is malicious, and D is a source of bytes to use for the occlusion of the original bytes. This approach allows us to find an approximate region of importance in only O(log jF j) calls to the detector C( ). In the event that C(Fl) = C(Fr), ties can be broken arbitrarily – which implicitly covers the boolean decisions from an antivirus product.
This method starts with a window size equal to half of the file size. Both halves are occluded and the file is analyzed by the classifier. The half that results in the largest drop in classification confidence is chosen to be split for the next time step. This binary search through the file is continued until the window size is at least as small as the target window size.
The last component of our approach is to specify what method D should be used to replace the bytes of the original file. One approach, which we call Random Occlusion, is to simply select each replacement byte at random. This is similar to prior work in finding the important regions of an image according to an object detector, where it was found that randomly blocking out the pixels was effective — even if the replacement values were nonsensical (Zeiler and Fergus 2014). We also look at Adversarial Occlusion, where we use a contiguous region selected randomly from one of our own
13:
size Require: A file F of length jF j, a classifier C( ), target occlusion size , byte replacement distribution D 1: split jF j=2, size jF j=2 2: start 0, end jF j 3: while size > do 4: Fl F , Fr F 5: Fl[split size:split] contiguous sample D 6: Fr[split:split+size] contiguous sample D 7: if C(Fl) < C(Fr) then 8: split split size=2 9: start split size, end split 10: else 11: split 12: start split + size=2 split, end split + size size=2 14: return start, end benign training files to occlude the bytes in the file under test. Since we use this approach only with malicious files, Adversarial Occlusion is an especially challenging test for our machine learning based methods: they have seen these byte regions before with an explicit label of benign. This test also helps to validate that all approaches aren’t simply detecting high-entropy regions that “look” packed, and then defaulting to a decision of malicious.
To show that our search procedure provides meaningful improvements for the Random and Adversarial Occlusions, we will also compare against an Undirected Occlusion approach that eschews our search procedure. In this case we randomly select a region of the binary to occlude with highentropy random bytes.
Recall, that this method identifies the critical region – for a successful attack the adversary would then have to manually alter the portion of the file at that location while maintaining their intended outcome. For this reason, we use a single target window size of 2048 bytes. The resulting occluded area will fall in the range of (1024-2048] bytes. On average, this restricts the occluded section to under 0.5% of the file size for our testing set. We also limit ourselves to searching for critical regions with our n-gram model. It would be infeasible for an adversary to conduct this search across all possible classifiers, and we would expect there to be some overlap among which regions are critical. This has been shown to be true in gradient based classifiers, where an adversarial example generated by one model can fool many others (Papernot, McDaniel, and Goodfellow 2016). 4.3
The last method we will consider is of a different variety. Instead of modifying known malicious files in an attempt to evade detection, we will inject malicious functionality into otherwise benign applications.
Many such techniques for this exist. We used the Return
Oriented Programming (ROP) Injector (Poulios, Ntantogian,
and Xenakis 2015) for our work. The ROP Injector converts
malicious shellcode into a series of special control flow
instructions that are already found inside the file. These
instructions are inherently benign, as they already exist in the benign
file. The technique patches the binary in order to modify the
memory stack so that the targeted instructions, which are
noncontiguous in the file, are executed in an order equivalent to
the original shellcode. The result is that the functionality of
the file is maintained, with the added malicious activity
executing right before the process exits. We use the same reverse
Meterpreter shellcode as Poulios, Ntantogian, and Xenakis
(
We note that not all benign executables in our testing set were injectable. The files must either have enough instructions to represent the shellcode already, or have room for further instructions to be added in a non-contiguous manner so as to prevent raising suspicion. Additionally, if the malware requires external functionality such as networking capabilities, and these are not inherent to the original executable, then portions of the PE header such as the import table must be adjusted. For these reasons, only 13,560 files were successfully infected.
Poulios, Ntantogian, and Xenakis claim they were able to evade anti-virus over 99% of the time using this technique. This approach should be nearly undetectable with static analysis alone, as all instructions in the file remain benign in nature. This makes it an extremely difficult test for both the machine learning and anti-virus detectors.
5
Before delving into results, we make explicit that we considered Malware the positive class and Benign the negative class . We remind the reader that not all tested systems are on equal footing. Our n-gram and MalConv models are trained on the same corpus, but we have no knowledge of the corpora used by the AV companies and their products. In all likelihood, they will be using datasets orders of magnitude larger than our own to ensure the quality of their products and to reach a desired false-positive rate. From this perspective alone, we would not necessarily expect the machine learning based methods to have better accuracies, as there is a disparity of information at training time. Our methods are also not optimized for the same low false-positive goal.
The accuracies and metrics of each method are presented in Table 1. Because of the mentioned information disparity between each system, we do not present this information for direct comparison. Our goal is to use these metrics as a base-line measure of performance for each method, and look at how these baselines are affected by various evasion techniques. 5.1
The results of the experiment described in subsection 4.1 are shown in Figure 1. All four anti-virus products were significantly degraded by making seemingly insignificant changes to the malware files. The machine learning models were immune to this evasion technique. Both model’s confidence that these files were malware changed by less than 1% on average. The few files that did evade were very close to the models’ decision boundaries before the modifications were performed.
AV3 AV4 1;000 s e l i F 400 800 600 200
0 g in 100 d a v E 80 s e iF 60 l f eo 40 g a tn 20 e c r eP 0 ram
v on alC M
The anti-virus products did not perform as well in this test. AV3 and AV4 were the most robust to these modifications, with 130 and 215 files evading each respectively. While this is better than products like AV1, where 783 out of 1000 malicious files where able to evade, it is still an order of magnitude worse than the 3 and 20 files that could evade the n-gram and MalConv models, respectively.
Given these results, one might wonder — could the ML based models be evaded by this approach if more modifications were performed? In particular, a number of the modifications have multiple results for a single action. So re-applying them is not unreasonable. To answer this question, we look at the evasion rate as a function of the number of modifications performed in Figure 2. In this plot we can see that all four AV products have an upward trending slope as the number of modifications is increased. In contrast, the n-gram model is completely unchanged from 3 or more modifications, and MalConv at 4 or more. This would appear to indicate that the ML approaches are truly immune to this evasion technique, whereas the AVs are not.
These results also speak to the use of dynamic evaluation, or lack thereof, in the AV products at scan time. Any dynamic analysis based approach should be intrinsically immune to the modifications performed in this test. Because we see all AV products fail to detect 13-78% of modified malware, we can postulate that if any dynamic analysis is being done its application is ineffectual.
We make note that in the original work of Anderson,
Filar, and Roth (
The experiment detailed in subsection 4.2 attempts to modify the most important region for detecting the maliciousness of a binary as deduced from querying only the n-gram model and attempting to transfer that knowledge to all others.
The results for the byte occlusion tests are given in Figure 3, where the blue bar shows the accuracy on 40,000 malicious files when no bytes are occluded. For both the n-gram and MalConv models, we see that the Random, Undirected, and Adversarial occlusion attacks have almost no impact on classification accuracy. In the case of the n-gram model, only 0.13% of the files where able to evade its detection after occlusion. Again evasions were highly correlated with files close to the decision boundary.
In particular, we remind the reader that the Adversarial occlusion is replacing up to 2KB of the malicious file with bytes taken from benign training data. This is designed to maximally impact the ML approaches, as the model was trained with the given bytes as having an explicit label of benign. Yet, the Adversarial choice has almost no impact on results. This is a strong indicator of the potential robustness of the ML approach, and that simply adding bytes from benign programs is not sufficient to evade their detection.
Considering the AV products we again see a different story. AV1 had the highest detection rate when no occlusion occurred, but also had the largest drop in detection rate for all three types of occlusion. AV4 had the best performance 4 s e l iF3 2
Random
Adversarial ram
v on alC M
From the AV results it is also clear that the Targeted Random occlusion is an improvement over the Undirected occlusion of random bytes, showing that Algorithm 1 is effective at identifying important regions. This was not discernible from the n-gram and MalConv models, which are relatively immune to this scenario.
Although on average the occlusion was limited to 0.5% of a file’s bytes, we acknowledge that one could argue this transformation may have altered the true label of the file. The search could be potentially removing the malicious payload of the application, rendering it muted (if runnable). We accept this possibility under the assumption that the adversary now has enough information to alter this small portion of the file in order to break possible signatures.
The results from the machine learning models would suggest that maliciousness is not contained to a small contiguous section of most files. This makes intuitive sense, as malicious functionality being employed in executable sections of a file might also require functions to be referenced in the import table. The non-contiguous nature of the PE file format allows for many similar circumstances, and so we believe it unlikely that all of a file’s maliciousness would be contained within a small contiguous region of a binary. In addition, this results in binaries that are analogous to deployed malware with bugs. Just because malware may not function properly, and thus not negatively impact users, doesn’t remove its malicious intent and the benefit in detecting it. In Table 2 we show the results of running the ROPInjector on the benign test files. The results are shown only for the 13,560 files that the ROPInjector was able to infect. Before infection, the Pre-ROP Accuracy column indicates the percentage of files correctly labeled benign. After infection, the Post-ROP Accuracy column indicates what percentage were correctly labeled as malicious. The Post-ROP Lift then shows how many percentage points of Post-ROP Accuracy came from the classifier correctly determining that a formerly benign file is now malicious, rather than simply being a false-positive. That is to say, if a model incorrectly called a benign file malicious, it’s not impressive when it calls an infected version of the file malicious as well.
Most malware detectors had significant difficulty with this task. The machine learning based models, n-gram and MalConv, showed only modest improvements of 0.4 and 1.2 percentage points. AV1 had a similarly small 0.6 improvement, but surprisingly AV2 and AV3 had negative improvements. That means for 1.4% of the benign files, AV3 called the original benign file malicious. But when that same file was infected by the ROPInjector, AV3 changed its decision to benign. This is a surprising failure case, and may be caused by the AV system relying too heavily on a signature based approach (i.e., a signature had a false-positive on the benign file, but the infection process broke the signature — turning a would-be true positive into a false-negative).
Overall these models showed no major impact from the ROPInjector. Only AV4 was able to significantly adjust its decision for 22.1% of the infected files to correctly label them as malicious. Though the evasion rate for AV4 remains high at 77%, it performed best in this scenario. Given the muted and negative performance of the other AV systems in this test, we suspect AV4 has developed techniques specifically for the ROPInjector approach.
We also notice that the benign files that were injectable include 67% of our original false positives for the n-gram model. We suspect that this is due to those files already containing the networking functionality required by the shellcode. The majority of malware requires communication over the Internet, therefore our machine learning detectors may view networking related features as somewhat malicious in nature.
Overall the ROPInjection of malicious code into otherwise benign applications presents an apparent weakness for our machine learning approaches. An important question is whether the ROPInjection is functionally invisible to the machine learning based models (i.e., it could never detect features of the injection), or is simply not sufficiently represented in our training data for the model to learn.
This question was tested by applying the ROPInjector to all of our training data, both benign and malicious files, which resulted in 235,865 total ROPInjected binaries. We then trained an n-gram model that tried to distinguish between ROPInjected vs Not-ROPInjected binaries. The n-gram model was able to obtain 97.6% accuracy at this task with an AUC of 99.6%. This indicates that the models could learn to detect ROPInjection, but that it is not sufficiently prevalent in our training corpus for the models to have learned.
Overall this case highlights a potential advantage of the classical AV systems with extensive domain knowledge. When sufficiently novel attacks are developed for which current models have almost no predictive power, it may be easier to develop and deploy new signatures to catch the attacks — while the machine learning approaches may require waiting for data, or creating synthetic data (which has its own risks) to adjust the model.
6
We have demonstrated a new testing methodology for comparing the robustness of machine learning classifiers to current anti-virus software. Furthermore, we have provided evidence that machine learning approaches may be more successful at catching malware that has been manipulated in an attempt to evade detection. Anti-virus products do a good job of catching known and static malicious files, but their rigid decision boundaries prevent them from generalizing to new threats or catching evolutionary malware. We demonstrated that top tier anti-virus products can be fooled by simple modifications including changing a few bytes or importing random functions. The machine learning models appear to better generalize maliciousness — leading to an increased robustness to evasion techniques compared to their anti-virus counterparts. sion and Counter-Evasion. In Proceedings of Reversing and Offensive-oriented Trends Symposium.