This paper studies the application of variational autoencoders (VAEs) to online learning from malware detection data. To this end, it employs a large real-world dataset of anonymized highdimensional data collected during 375 consecutive weeks. Several VAEs were trained on selected subsets of this time series and subsequently tested on diferent subsets. For the assessment of their performance, the accuracy metric is complemented with the Wasserstein distance. In addition, the influence of diferent kinds of data normalization on the VAE perfomance has been investigated. Finally, the combinations of a VAE with two multi-layer perceptorns (MLPs) have been investigated, which has lead to the surprising result that the impact of such a combination on malware detection is positive for a simple and superficially optimized MLP, but negative for a complex and well optimized one.
mance, versatility, scalability and learning of features. Due to feature learning, a neural network, especially a deep one generally does not need extensive feature engineering. On the other hand, it requires a large amount of training data. There exist application areas where feature distribution and optimal classification can change in time. Classification in such situations is often called online classification. One of such areas is malware detection. The reason for changing distribution in malware detection is data drift, which comes from several sources. Firstly, benign programs are using diferent frameworks or statically linked libraries based on changing popularity. Secondly, malicious programs attempt to hide by copying clean program’s code and behaviour, or using evading techniques like the manipulation of code during runtime, and those techniques are also changing in time, as a reaction on updating the databases of malware detection software. For this reason, neural networks hardly ever have enough training data to be promptly prepared for new threats. Moreover, even if they had enough data, their training would probably be too slow. This paper reports a work in progress investigating one of the possibilities to adapt neural networks for malware detection. We prepared a hybrid model that consists of multiple pairs of a generator and a classifier. Our idea is to make the generators learn the history and later use them for data augmentation because many malware evasion techniques are shared. As the generators, we use variational autoencoders (VAEs), and classification is performed using multi-layer neural networks. Firstly, we present results of our hybrid classification-autoencoder model, which employs a kind of transient feature learning.
when it is trained with high amounts of outliers.
2
Online Malware Detection
vulnerabilities and examining evading techniques [
and consequently complicates the application of classification techniques for malware detection. These techniques must be precise enough not to bother user with false positives, as well as highly adaptive in order to detect new threats as soon as possible.
Malware detection techniques are static and
dynamic [
proach to malware detection is DroidOL [
tation is robust against hiding attempts of malicious software. After WL prepossessing, DroidOL applies a passive-aggressive classifier, which is a kind of online learning methods suited for big data. On real
were investigated in [
A diferent approach to malicious software was
investigated in [
main idea is firstly introduced, now we try to
identify the problems we had and try to come up with
a solution that could be useful in similar situations.
We are especially interested in the difficulty of VAE
training since it is a model which is usually applied to
problems where pictures are involved. Pictures have
really nice property – all features have limited,
similarly important values. It could also be the reason
why VAE usually smoothes generated samples [
Autoencoders are artificial neural networks capable of nonlinear space reduction. They are trained almost like classical neural networks for regression, but their objective is the reconstruction of their input. The reconstruction function is any loss function that could be used for regression problems. A usual choice is the mean squared error (MSE) or the mean absolute error (MAE). Their main benefit comes from the introduction of a bottleneck. The bottleneck can be a narrow part of the network or some heavy regularisation. Such a bottleneck is required to have a restricted lfow of information. This restriction causes an autoencoder to preserve only the most important patterns in data and drop that kind of information that can be easily deduced or is noisy. An interesting fact is that neural networks with identity activation function and
cipal component analysis transformation [
commonly, they are used as a dimensionality
reduction method, or for denoising – where the input is
formed by corrupted image while a crisp image is
expected as an output [
Variational Autoencoder (VAE)
on a small number of real samples. For these types of problems, autoencoders ofer a solution. Bottlenecks significantly reduce chances for overtraining while maintaining appropriate power due to nonlinear space reduction.
a new sample, we need to know the distribution of codings. For this reason, VAE adds two updates to regular autoencoder that makes codings obey some distribution (usually normal). The first update is to add another term in the model’s loss. The Kullback
distributions m and n and is defined by
DKL(m k n ) = H(m ;n )
Z
Rn = m (x) lnn (x)dx +
m (x) ln m (x)dx H(m )
Z Z
Rn
Rn = m (x) ln m (x) n (x) dx; where H(m ;n ) is cross-entropy and H(m ) is entropy. If m and n equals, the divergence is 0. Otherwise, it is a positive value.
The model can link small changes in codings with large changes in result, giving an opportunity to overtraining. So the second update is to add uncertainty into codings. In practice, the coding layer of VAE is split into two branches, one for means and one for variances. The second part of the network – decoder then receives samples based on these parameters of the generating distributions. In fact, it is common to use the logarithm of variance since it has a more suitable range. Then a loss based on the KL divergence can be reduced to LVAE = LReconstruction 1 N G
å å 1 + vil 2N i=1 l=1
2
mil
evil
where N is the batch size, G is the coding dimension,
vi j is the coding variance and mi j is the coding mean
[
Let k k be an arbitrary norm on Rn; n 2 N and m ;n be
probability measures on Rn with finite first moments.
For each n-dimensional random vector X , denote
D (X ) the distribution of X . Then the Wasserstein
distance of order 1, or simply Wasserstein distance
(aka Wasserstein measure, Kantorovich-Rubinstein
distance, earth mover’s distance) between m and n
corresponding to the norm k k is defined [
Y k j D (X ) = m ^ D (Y ) = n g : (1)
a method for model comparison. Unlike Kullback
ric and fulfills the triangle inequality. Moreover, it
does not require the considered measures to share a
probability space. The Wasserstein distance is also
scale sensitive W (cm ; cn ) cb W (m ;n ) and sum
invariant W (m + A;n + A) W (m ;n ) for any b ; c > 0 and
A 2 Rn [
Considered kinds of normalization
ized with 14 diferent transforming functions, denoted
T1 T14. The definitions of nearly all of them are quite
simple, therefore, they are merely listed in Table 1.
The only exception is T5, which performs a power
transformation proposed by Yeo and Johnson [
We use the following notations: a 2 R denotes an arbitrary value of a transformed attribute A, whereas A denotes the set of values of A in the training data a1; : : : ; ap, kAk denotes the set of absolute values of A in ka1k; : : : ; kapk, Aq; q 2 (0; 1) denotes the quantile q of a sample of A in the training data, f is the cumulative distribution function of the standard normal distribution N(0; 1), and e is the Euler constant, on which natural logarithms are based. Because T6; T7; T13; T14 are based on quantiles prediction, the final transformations are formed by substitute linear interpolations for those quantiles. Due to performance reasons, the quantiles are predicted based on 100000 samples of A.
I. A parametrized version y of the intended transformation is defined, which depends on a parameter l 2 R: y (a;l ) = 8 (a+1)l 1 >>> l ><ln(a + 1)
( x+1)2 l 1 >>> 2 l > : ln( x + 1) if a 2 R 0;l 6= 0; if a 2 R 0;l = 0; if a 2 R<0;l 6= 2; if a 2 R<0;l = 2: (2) II. It is assumed that for some range of values of the parameter l , the value of y (a;l ) is a random variable with the distribution N(m ;s 2) for some m 2 R;s 2 R>0. Consequently, the log-likelihood of the parameters (l ; m ;s 2) with respect to training data is `(l ; m ;s 2; a1; : : : ; ap) = + (l p 2 logs 2
p 1 2s 2 iå=1(y (ai;l )
m )2 p 1) å sign(ai) log(kaik + 1): i=1 logs 2(l ):
4.2 (3) III. The parameters m and s 2 are for each l 2 R estimated, respectively, with the following estimates mˆ (l ) and sˆ 2(l ): mˆ (l ) = sˆ 2(l ) = 1 p
å y (ai;l ); p i=1 1 p
å(y (ai;l ) p i=1 mˆ (l ))2: IV. For the parameter l , the maximum likelihood estimate is used, after replacing the other two parameters by their estimates: lˆ = max `(l ; mˆ (l );sˆ 2(l ); a1; : : : ; ap) = l 2R
p max(l 1) å sign(ai) log(kaik+1) l 2R i=1
We cannot measure the Wasserstein distance in transformed space because there is no clear way of comparing the results between transformations. It is not a problem for most of the transformations we use, but the perc-uniform and perc-normal are slightly problematic. We have to use a linear interpolation on a sequence of quantiles in order to approximate the initial dataset. For this reason, we expect to have some increase in the Wasserstein distance on perc variants since it is nonzero value even if we compare original dataset to its forward and then backward transformed version. 4 4.1
Experimental Evaluation
malware and clean software in several categories, but we consider only two by merging some of them. Antimalware software producers must protect their knowhow by providing only data which were anonymized. For us, the anonymization process is unknown. It essentially hides the meaning of features in provided datasets by stripping it in the documentation and applying unknown functions on those columns. The feature space is very complex, there are 540 features with various distributions. This makes particularly difficult to choose the correct data scaling. In Figure 2, several kinds of features are diferentiated: • Binary feature • Gaussian feature: both absolute skewness and excess kurtosis are less than 2 • Highly skewed feature: skewness > 30 • Almost constant feature: more than 99.9 % values are identical • Other unknown distributions
20% 30%
Our dataset has many outliers. We were interested in how outliers efect generator. Therefore, we compared two common loss functions used for regression, both combined with the kinds of normalization considered in Subsection 3.4. The first was the mean square error (MSE) and the second was the mean absolute error (MAE). Both are tested on a rather large network with 7 layers [541; 306; 173; 98; 56; 32; 18; 10], with ELU as activation function and batch normalisation on every layer except the coding and output one. The results can be seen in Figure 4 It looks like the best normalisation for our data is the power transform, but the diferences between diferent kinds of normalizations are, in general, small. It transforms data to be more normal. Both perc-uniform and percnormal performed excellently. It seems that small distortions created by linear interpolations of quantiles do not deteriorate result.
and max-abs types of normalisations, especially if MSE is employed. We observe that some of the features consist of a small-valued majority with some huge values. Min-max and max-abs transformations could make this range very small, demanding high precision on regression for the smaller ones. On the other hand, the robust transformation could make the range larger because quantities could be evaluated only on the small values – artificially increasing existing variance.
It does not seem that the log variants have any positive efect, but we saw a slight increase in min-max and max-abs transformations. This is expected because logarithm reduces ranges of values. Robust transformation has a substantial incompatibility with logarithm transformation because log-robust transformation operates on logarithmic space. After exponentiation, the errors could be much larger. However, this explanation fails if we consider that log-perc transformations performed almost equally.
in perc-uniform, perc-normal, log-perc-uniform, logperc-normal, power, log-power.
week 1 week 2 week 3 VAE
VAE
VAE MLP
MLP
MLP
ing size afect the resulting performance. It seems there is no diference, 10 as a coding dimension (predicted normal distributions of 2 parameters) is sufficient no matter which normalisation was chosen. The results can be seen in Figure 5.
Figure 6 shows expected behaviour. The VAE has trouble expressing features with higher variance. It is evident, especially on the highest tertile, where diferences between distributions are almost non-existent.
A next view on the data reveals something interesting – there is a great diference between the data with diferent signs of skew of a particular feature. Negative skew was much harder to imitate while positive skew was easier to reproduce than in the other two middle quartiles. This could be a property of dataset features or under-training. It seems that max-abs and min-max normalizations have a slightly diferent response. 4.3
Experiments with Diferent MLPs We were interested in the ability of generative models to capture important information (e.g. vanishing or strange behaviour) and ignore the noise (waiting for user input). As the next experiment, we prepared a balanced dataset extended by a feature reporting true benign and malware labels. Firstly, we let the generative model learn on the first part of data without true labels. Then, every other part of the data was augmented by classified features generated by the previous generator and classified by the previous classifier.
of MLPs hyperparameters. The first, MLP 1 is the result of mid-optimisation and MLP2 is the final result.
thought the MLP1 is significantly worse than MLP2 in 94% of weeks, when VAE was added, it became significantly better on 18% of weeks while MLP 2 on none them.
methods can be seen. The VAE with the MLP1 is significantly better than the VAE with the MLP 2 for the weeks 1-71. This is interesting because a separate MLP1 trained on a small amount of samples is generally worse than MLP2.
Firstly, the hyperparameters of MLP2 can be simply overtrained on training samples. Instead of learning about patterns, the model may learn small mistakes made by VAE and try to infer based on them. Secondly, MLP1 may lack some form of regularization. 5
autoencoders to online learning from malware detection data. To this end, it employed a large realworld dataset of anonymized high-dimensional data.
pairs of a generator and a classifier. The basic idea of our approach is to make the generators learn the history and later use it for data augmentation. In addition, the influence of diferent kinds of data normalization on the VAE perfomance has been investigated.
been complemented with the Wasserstein distance.
The experiments have proven our expectation that for highly heterogeneous data, normalization is relevant. However, the most basic transformations like min-max and max-abs were worse than using the raw value without any transformation. The experiments have also shown that variational autoencoders can be used for data augmentation in MLP-based classification. However, one should be aware of the hyperparameters determining the size of the classifying MLP.
ported by the Czech Science Foundation (GAČR) grant 18-18080S. Computational resources were supplied by the project ”e-Infrastruktura CZ” (e-INFRA
lo g - m lo g - m ax-abs in -m lo gax lo gpow er lo gperc-no rm lo gperc-unifo al rm robust lo gstand.
m ax-abs m in -m ax pow er perc-no rm perc-uniforaw al rm robust stand. robust lo gstand.
m ax-abs m in -m ax pow er perc-no rm perc-uniforaw al rm robust stand. robust log- perc-normal log- perc-uniform log- robust log- stand.
max-abs min-max power perc-normal perc-uniforramw robust stand.
Optimal Transport, Old and New.