=Paper= {{Paper |id=Vol-3095/paper5 |storemode=property |title=Rank-1 Similarity Matrix Decomposition For Modeling Changes in Antivirus Consensus Through Time |pdfUrl=https://ceur-ws.org/Vol-3095/paper5.pdf |volume=Vol-3095 |authors=Robert J. Joyce,Edward Raff,Charles Nicholas }} ==Rank-1 Similarity Matrix Decomposition For Modeling Changes in Antivirus Consensus Through Time== https://ceur-ws.org/Vol-3095/paper5.pdf

Rank-1 Similarity Matrix Decomposition For Modeling
Changes in Antivirus Consensus Through Time
Robert J. Joyce∗1,2 , Edward Raff†1,2 , and Charles Nicholas‡2
1
Booz Allen Hamilton
2
University of Maryland, Baltimore County

Abstract
Although groups of strongly correlated antivirus engines are known to exist, at present
there is limited understanding of how or why these correlations came to be. Using a cor-
pus of 25 million VirusTotal reports representing over a decade of antivirus scan data, we
challenge prevailing wisdom that these correlations primarily originate from "first-order"
interactions such as antivirus vendors copying the labels of leading vendors. We introduce
the Temporal Rank-1 Similarity Matrix decomposition (R1SM-T) in order to investigate the
origins of these correlations and to model how consensus amongst antivirus engines changes
over time. We reveal that first-order interactions do not explain as much behavior in an-
tivirus correlation as previously thought, and that the relationships between antivirus en-
gines are highly volatile. We make recommendations on items in need of future study and
consideration based on our findings.

1 Introduction
Our work is motivated by two chronic problems in the study of malware, namely malware
detection (deciding whether a file is benign or malicious) and malware family classification
(determining which of many existing families a malware sample might belong to). These tasks
both require labeled data, but new malware samples number in the millions each month [19] and
obtaining ground truth labels via manual analysis can take hours of effort per sample [24]. For
this reason, the vast majority of works use the aggregated results from a collection of antivirus
engines as a source of scalable labeling [26]. For example, a common approach to malware
detection is antivirus thresholding, in which some minimum number of antivirus engines in a
collection must detect a file as malicious in order for it to be considered malware [5, 8]. Likewise,
plurality and majority voting amongst antivirus engines are popular strategies for performing
malware family classification [17, 2]. A significant issue with these aggregation approaches is that
all antivirus engines are treated as independent voters, yet prior work shows that some groups
of antivirus engines make highly correlated labeling decisions [9, 26, 17]. As is well attested
within the ML literature, the use of highly correlated models provides little benefit [7, 25, 3].
The presence of strong correlations between some antivirus engines likely results in degraded
accuracy when these voting methods are used.
Although the existence of correlations between antivirus engines is well-documented, there
has been minimal study of why they exist. Present explanations include different engines created
by the same company, products “copying” the results of leading vendors, and vendors sub-
licensing their technology to others [17, 12]. All of the above explanations can be considered
∗
joyce_robert2@bah.com
†
raff_edward@bah.com
‡
nicholas@umbc.edu

Copyright © 2021 for this paper by its authors. Use permitted under Creative Commons
License Attribution 4.0 International (CC BY 4.0).
Proceedings of the Conference on Applied Machine Learning for Information Security, 2021
R1SM Decomposition For Modeling Antivirus Consensus R. Joyce, E. Raff, and C. Nicholas

“first-order” interactions, since they create a direct link between the labeling decisions of two
antivirus engines. To our knowledge, no existing work has empirically confirmed whether first-
order interactions are the sole cause of the correlations between antivirus engines, or whether
more complex, unknown factors are also (at least in part) responsible.
An additional consideration overlooked by prior work is the volatile and adversarial nature
of the malware ecosystem. Malware authors are constantly attempting to evade detection while
antivirus engines are continually forced to develop new detection methods [13]. We hypothesize
that the groups of antivirus engines which are highly correlated may themselves change as a
function of time. However, we are aware of no prior work which has studied this possibility [14].
Our work does not attempt to explain how correlations between antivirus engines came to
exist, but instead seeks to answer questions about the nature of these correlations and how they
change over time. In Section 2 we discuss the current state of research on antivirus engine dy-
namics. In Section 3 we explore consensus amongst antivirus engines and how it has changed
over the course of a decade. In Section 4 we introduce the Rank-1 Similarity Matrix decom-
position (R1SM), which reveals first-order interactions between the constituents of a similarity
matrix. The section also discusses an extension to R1SM that uses a neural network over po-
sitional embeddings to concurrently decompose a time-series of similarity matrices, which we
term R1SM-T. In Section 5 we apply R1SM and R1SM-T to over 25 million antivirus scan
reports spanning a decade in order to identify first-order interactions between the constituent
antivirus engines. Our results indicate that relationships between antivirus engines are more
mercurial than previously thought. Finally, in Section 6 we discuss the impacts of our findings
and conclude that future antivirus aggregation strategies should consider approaches similar to
a weighted ensemble, where the weights of each antivirus engine are a function of time.

2 Related Work
Mohaisen and Alrawi [12] is the earliest work we are aware of which systematically evaluates
the performance of antivirus engines. The authors observed that the detection results of many
antivirus engines follow those of a leading product and hypothesize that this correlation is due
to copying or sharing of information. Hurier et al. [6] introduced several metrics for quantify-
ing the level of consensus between a set of antivirus engines. In Section 3.3 we explore how one
of these metrics, synchronicity, changes over a ten year period. Kantchelian et al. [9] observed
that antivirus labels take time to stabilize and that vendors may change their detections to cor-
rect errors, especially false negatives. In a study of 734,000 executables first seen on VirusTotal
(an online malware analysis service that scans files with a collection of antivirus engines) be-
tween Jan. 2012 and Jun. 2014, the authors measured correlation amongst the detections of a
group of approximately 80 antivirus engines. They found that although some groups are highly
correlated, antivirus engines lack an overall consensus. Martín et al. [11] surveyed a dataset of
82,866 suspicious Android applications and showed that some antivirus engines also make cor-
related decisions when labeling malware as a particular category or family. The closest work to
ours is Zhu et al. [26], who re-scanned a collection of 14,000 malware samples daily for over a
year in order to investigate the dynamics of antivirus detection changes. By observing which
antivirus engines changed their detections with similar timing, the authors identified five groups
of highly correlated antivirus engines. Furthermore, Zhu et al. [26] used influence modeling to
identify antivirus engines which actively change their detections to match other vendors. They
determined that label copying is a widespread practice in the antivirus industry. All of these
works generally lead to first-order conclusions about correlation, but do not study the correla-
tions on the same quantity of data (25 million scan reports) or length of time (ten years) that
we consider in this study.

Proceedings of the Conference on Applied Machine Learning for Information Security, 2021
R1SM Decomposition For Modeling Antivirus Consensus R. Joyce, E. Raff, and C. Nicholas

3 Studying Changes in Antivirus Engine Consensus Through Time
For the purposes of investigating antivirus label consensus and how antivirus dynamics have
changed through time, we were provided with a dataset of 25,100,286 VirusTotal scan reports
[18]. This dataset, which we call VirusShare-VT, was collected by querying the VirusTotal API
for all files in chunks 0 through 233 of the publicly-available VirusShare malware corpus [1].
VirusTotal API queries for the VirusShare-VT dataset were made over the course of six months,
from Dec. 2015 to May 2016 [18]. Each report in the VirusShare-VT dataset is a JSON object
containing information about a particular VirusTotal scan. Of note is the scan_date field, which
contains the date and time that a file was scanned with the collection of antivirus engines. The
scan date is often older than the query date, because VT does not re-scan files for simple queries.
The distribution of scan dates is shown in Figure 1, ranging from May 2006 to May 2016.

Figure 1: Distribution of scan dates in VirusShare-VT.

Given the sizeable number of malware samples in chunks 0 - 233 of VirusShare, scanning these
samples daily as Zhu et al. [26] did was infeasible. The VirusShare-VT dataset only contains
one scan report per sample, and antivirus detections for files first seen shortly before the scan
date have likely not stabilized. However, we do not consider these factors to be drawbacks, as
they would be typical of most datasets used for antivirus aggregation. The massive size and
timescale of the VirusShare-VT dataset makes it ideal for answering our research questions.

3.1 Measuring Pairwise Antivirus Consensus
Throughout this paper we attempt to follow the terminology introduced by Hurier et al. [6]
for measuring consensus amongst antivirus engines. Given a set of n antivirus engines A =
{a1 , a2 , ..., an } and a set of m files P = {p1 , p2 , ..., pm }, the detections and family classifications
of the antivirus engines for this set of malware samples can be arranged into two matrices B
and C:
   
b1,1 b1,2 . . . b1,n c1,1 c1,2 . . . c1,n
 b2,1 b2,2 . . . b2,n   c2,1 c2,2 . . . c2,n 
B= . . .. . C =  .. .. .. .. 
   
 .. .. . ..   . . . . 


bm,1 bm,2 . . . bm,n cm,1 cm,2 . . . cm,n

An element Bi,j in B is 1 if file pi is detected as malware by engine aj and 0 if it is not
detected. An element Ci,j in C is given by the malware family assigned to file pi by engine aj . Di,j
and Ci,j are ∅ (null) if engine aj did not scan pi . For constructing the matrix C we employed a
portion of the AVClass labeler’s architecture, which can extract family information from antivirus
signatures [17]. When AVClass ingests a scan report, it normalizes and tokenizes each antivirus
signature, removes any tokens that do not contain family information, and performs family alias

Proceedings of the Conference on Applied Machine Learning for Information Security, 2021
R1SM Decomposition For Modeling Antivirus Consensus R. Joyce, E. Raff, and C. Nicholas

resolution. The processed token(s) from the antivirus signature produced by engine aj for file
pi are used as the family for element Ci,j .
Hurier et al. [6] proposed a metric called overlap for computing pairwise detection consensus
for a pair of antivirus engines. However, overlap does not consider that some antivirus engines
may be missing from a scan report. Instead, we define a similar metric, which we call agreement,
that corrects this issue.
|Bi ==
S Bj |
Definition 1. Agreement(Bi , Bj ) = |Bi Bj | s. t. Bi , Bj ̸= ∅

Agreement(Bi , Bj ) divides the number of scans in B in which ai and aj agree upon a file’s
detection by the total number of scans in which both ai and aj are present. Classification
agreement can be defined in the same way by substituting the matrix B for C. Since it is possible
for AVClass to convert a single antivirus signature into multiple family tokens, we consider two
elements in C to be equal if they share any AVClass tokens, or if AVClass produced zero tokens
for both signatures.

3.2 Antivirus Agreement in VirusShare-VT
The VirusShare-VT dataset contains 93 antivirus engines that appear in at least 1,000 different
scan reports. The set of antivirus engines used by VirusTotal changes gradually over time. In
May 2006 only 26 of the 93 engines were observed; this number gradually increases to 57 by May
2016. The sets of antivirus engines in VirusTotal are relatively consistent month-to-month, with
an average of 1.033 engines added or removed per month. Several antivirus engines only appear
in VirusShare-VT during a short window of time. Many of these are alternative or beta versions
of existing engines (e.g. PandaBeta from Feb. 2007 to Feb. 2009, McAfee+Artemis from Nov.
2008 to Jan. 2011, and Avast5 from Mar. 2010 to Sep. 2011). The name, numeric index used
in all appropriate figures, and total number of occurrences of each of the 93 antivirus engines in
the VirusShare-VT dataset is shown in Table 1 in Appendix A.

(a) Detection Matrix (b) Classification Matrix

Figure 2: Similarity matrices displaying pairwise detection and classification agreement for 93 antivirus
engines in VirusShare-VT.

Figures 2(a) and 2(b) show the pairwise detection and classification agreement for each of
these 93 antivirus engines. Consistent with prior work, there are observable instances of high
detection consensus among some vendors, and a small subset of vendors have very little agree-
ment with others [9]. The classification agreement matrix appears highly similar in structure
to the detection matrix but with smaller values on average. One possible explanation for this
phenomenon is that classification agreement depends upon both antivirus engines detecting the
sample as malware.

Proceedings of the Conference on Applied Machine Learning for Information Security, 2021
R1SM Decomposition For Modeling Antivirus Consensus R. Joyce, E. Raff, and C. Nicholas

3.3 Measuring Changes in Antivirus Synchronicity Over Time
Next, we explore how overall consensus amongst antivirus engines has changed over time. Con-
sider a similarity matrix D constructed
P by applying some similarity function sim(Bi , Bj ) to each
pair of antivirus engines in A. Let D denote the sum of all elements in D. Because values
below the main diagonal of a similarity matrix are redundant, we define the triu(X, i) function
to return X where all elements at or below the ith diagonal are replaced with zero. In future
references to similarity matrices in this paper it is implicit that redundant information has al-
ready been removed, i.e. D has been replaced with triu(D, 1). To measure overall consensus
amongst a set of antivirus engines, we use synchronicity, defined as [6]:
P
triu(D,1)
Definition 2. Synchronicity(B) = n(n−1)/2

Synchronicity is equivalent to the average value of the entries above the main diagonal of D.
We define synchronicity using different notation than Hurier et al. [6] to to be consistent with
terminology we use later in this paper. When computing the similarity matrix D, sim(Bi , Bj )
can be any pairwise similarity function; we elect to use agreement as this similarity function in
all of our experiments. Although Hurier et al. [6] define synchronicity only for measuring the
level of consensus amongst antivirus detections, it can also measure classification consensus by
computing a similarity matrix for C instead of B.

3.4 Monthly Synchronicity in VirusShare-VT
Figure 3 displays how the synchronic-
ity of the antivirus engines in the
VirusShare-VT dataset changes over
time. This data was collected by group-
ing the scans in VirusShare-VT by month
and computing detection and classifi-
cation synchronicity for each group of
scans. It is evident that synchronicity
amongst antivirus engines varies consid-
erably over short spans of time. Although
they have different magnitudes, detection
and classification synchronicity seem to
be loosely correlated. Again, a possible
explanation for this is because classifica-
Figure 3: Monthly detection and classification synchronic-
tion must follow detection. ity amongst antivirus engines in VirusShare-VT.
At first, we believed that one factor
which contributed to the volatility shown in Figure 3 was engines joining and leaving the Virus-
Total platform. As we mentioned in Section 3.1, changes in the set of antivirus engines used by
VirusTotal tend to be very gradual. However, we identified three events in which four or more
antivirus engines were added or removed in the span of a month. One of these represents the
most significant population shift in our dataset by far; the removal of fourteen engines between
Jan. and Feb. 2009. This corresponds to an increase in detection synchronicity from 0.577 to
0.679 during this period, though change in classification synchronicity is negligible. The other
two events are the additions of four antivirus engines between Aug. and Sep. 2008 and five en-
gines between Aug. and Sep. 2013. However, synchronicity does not change significantly during
either of these intervals. It would be difficult for changes in synchronicity to occur due to popu-
lation changes unless a significant number of engines join or leave. In addition, we observe other
significant increases and decreases in synchronicity during which the population of antivirus en-
gines does not change. We conclude that changes in synchronicity amongst antivirus engines

Proceedings of the Conference on Applied Machine Learning for Information Security, 2021
R1SM Decomposition For Modeling Antivirus Consensus R. Joyce, E. Raff, and C. Nicholas

are likely caused by a complex assortment of factors, including changes in both the malware
ecosystem and antivirus community.

4 Temporal Rank-1 Decomposition of Similarity Matrices
All current explanations for consensus between antivirus engines can be classified as first-order
interactions, i.e. a single interaction between a pair of features. In order to test these widely-
held assumptions we introduce the Rank-1 Similarity Matrix (R1SM) decomposition. We later
describe an extension to R1SM that reveals changes in first-order interactions within time-series
data, which we call the Temporal Rank-1 Similarity Matrix Decomposition (R1SM-T). It was
necessary to create this decomposition as, to our knowledge, no existing algorithm possesses this
capability.

4.1 The R1SM Decomposition
Suppose we have a similarity matrix D that represents agreement between each pair of antivirus
engines in A. The R1SM decomposition exposes first-order interactions between the antivirus
engines in the upper triangular of D as the sum of rank-1 outer products with shared, non-
negative weights.
Definition 3. D = ki=1 triu(r i r ⊤
P
i , 1)

In Definition 3, each vector r 1 , r 2 , ...r k has length n and is non-negative. First-order inter-
actions between objects in D manifest in these vectors, which we call the components of the
decomposition. This behavior occurs due to the nature of the decomposition, in which the outer
product of each vector ri and its transpose forms a rank-1 matrix (a matrix containing only
first-order interactions by definition). The R1SM decomposition is comparable to the existing
CANDECOMP/PARAFAC (CP) decomposition, which also decomposes a tensor into a sum of
rank-one outer products [10]. However, additional restrictions (e.g. the decomposition can only
be applied to the upper triangular of a square, non-negative matrix and the rank-one outer prod-
ucts have shared weights) distinguish the R1SM decomposition from the CP decomposition.

4.2 Solving the R1SM Decomposition
Next, we discuss how the R1SM decomposi-
Algorithm 1 R1SM Greedy Decomposition
tion is computed. A trivial solution of the
R1SM decomposition exists for all similarity Require: Similarity matrix D, early stopping
matrices in which each component determines threshold δ
1: function R1SM-Greedy(D, δ)
a single value in one of the n(n−1)/2 elements
in the upper triangular. However, this solu- 2: Y1 ← D, i ← 0
tion does not provide any useful insights about 3: do
first-order interactions in the decomposed ma- 4: i←i+1
trix. Recall that one of our research goals is 5: Find ri which maximally explains Yi
to determine what portion of the correlations 6: Ri ← triu(r i r ⊤ i , 1)
can be explained by first-order interactions. 7: Yi+1P← Yi − Ri
while P Ri ≥ δ
In order to obtain this information, we solve 8: D
for the components of the R1SM decomposi- 9: return r 1 , r 2 , ...r i−1
tion using an iterative, greedy strategy.
Algorithm 1 approximates the R1SM decomposition of a similarity matrix D. At the begin-
ning of the ith iteration of the algorithm, Yi is the residual of triu(D, 1), representing the portion
of the similarity matrix that has not yet contributed to the decomposition. At each step of the
decomposition, a component ri is found such that ri maximally explains Yi , i.e. the maximum
value of Ri for which Yi − Ri is non-negative, where Ri = triu(r i r ⊤ i , 1) (line 5). In Section
P

Proceedings of the Conference on Applied Machine Learning for Information Security, 2021
R1SM Decomposition For Modeling Antivirus Consensus R. Joyce, E. Raff, and C. Nicholas

4.4 we describe our implementation for finding components that maximally explain Yi . After
solving for ri , the updated residual Yi+1 is computed by subtracting Ri from Yi (line 7).
Each Pcomponent of the R1SM decomposition explains a portion of the similarity matrix,
given by PRDi . Due to the greedy nature of Algorithm 1, the percentage of the similarity matrix
explained by subsequent components tends to decrease monotonically. Once a component fails
to explain a meaningful percentage of the similarity matrix, it is unlikely that any subsequent
component will. Once the algorithm reaches this point, we assert that most if not all significant
first-order interactions have been captured by the decomposition, and all further information
left to be explained is better represented by a morePcomplex model. Therefore, iteration of
Algorithm 1 halts if a component is found for which PRDi is less than δ, which defaults to 0.1%
(line 8). If a significant portion of D can be decomposed before the early stopping condition is
reached, we conclude that most of the interactions between the antivirus engines represented by
D are first-order. A complete decomposition of D can be obtained by setting δ to zero, in which
Algorithm 1 will iterate until triu(Yi , 1) stores the zero matrix.

4.3 R1SM Cluster Extraction
Each component r i of the R1SM composition represents first-order interactions between objects
in a similarity matrix. As such, each component can be interpreted as a cluster, where large val-
ues in a component indicate a strong first-order relationship between the corresponding objects.
Unlike traditional methods for clustering objects in a similarity matrix (e.g. agglomerative hi-
erarchical clustering), which group objects by their overall similarity, the clusters produced by
the R1SM decomposition indicate groups with prominent first-order interactions. We stress that
clustering is not the primary motivation of the R1SM decomposition, but we explore the idea
due to its usefulness.
Because the r i are not sparse, they may contain small, even spurious values that are not
indicative of significant first-order interactions between objects. Thus a parameter ϵ influences
which members of a component are considered “clustered” (i.e., a non-trivial first-order correlate).
For a component r i , the j’th object is a member of cluster i iff r ij ≥ ϵ. A large ϵ results in
smaller clusters, where all objects within a cluster have strong first-order interactions between
each other. Conversely, a small ϵ yields larger clusters, but objects within a cluster may have
weaker first-order interactions. An object may be a member of multiple clusters or none at all,
and it is possible for a cluster to contain zero objects. The early stopping term δ also controls the
resulting clustering, as it determines the maximum number of clusters. In Section 5.1 we take
advantage of the clustering property of the R1SM decomposition to identify groups of antivirus
engines that share strong first-order interactions.

4.4 RISM-T: Applying the R1SM Decomposition to Time-Series Data
One of our primary research goals is to study how first-order interactions amongst antivirus
engines have changed over time. In order to do so, we introduce an extension to the R1SM
decomposition which can decompose a time-series of similarity matrices D = [D1 , D2 , ...DT ]
rather than a single matrix. We call this extension the Temporal Rank-1 Similarity Decomposi-
tion (R1SM-T). Again, it is implicit that all similarity matrices in the time-series have had the
redundant information at or below their main diagonals replaced with zero.
Algorithm 2 describes the concurrent R1SM decomposition of multiple similarity matrices
while sharing information across all matrices as a function of their spatial relationships in time.
During the ith iteration, Y i = [Yi,1 , Yi,2 , ...Yi,T ] stores the residual of each similarity matrix in D.
Like Algorithm 1, components ri = [r i,1 , r i,2 , ...r i,T ] are found such that they each maximally
explain their respective matrices in Y i (lines 6 - 13). Our implementation for finding these
components is described momentarily. A penalty term discourages any values in triu(r i,t r ⊤ i,t , 1)
from exceeding their corresponding values in Yi,t , but minute errors are still possible. Therefore,

Proceedings of the Conference on Applied Machine Learning for Information Security, 2021
R1SM Decomposition For Modeling Antivirus Consensus R. Joyce, E. Raff, and C. Nicholas

for each time t, triu(r i,t r ⊤
i,t , 1) is corrected using Yi,t and the result is stored in Ri,t (line 16).
Yi+1,t , the new residual of triu(D, 1), is computed by subtracting R Pi,t
from Yi,t (line 17). Like
Ri
Algorithm 1, iteration stops once components are found such that D < δ (line 18).
P

Our implementation uses a deep neural network F (·) over positional embeddings to concur-
rently solve the next component in the R1SM decomposition for each similarity matrix in the
time-series. This model design was selected so that non-linear changes in consensus over time
can be learned. Furthermore, positional embeddings allow the model to leverage temporal rela-
tionships between the target similarity matrices as the primary factor of changes in consensus.
F (·) is trained on a batch of input vectors X = [X1 , X2 , ..., XT ], where each vector Xt is the
positional embedding of timestep t in the time-series. To obtain the positional embedding of t,
we define d2 distinct frequencies f1 , f2 , ...fd/2 , where d is the size of the neural network’s input
layer and the j th frequency is given by fj = t
2j . Xt is constructed by alternately applying
10000 d
the sin() and cos() functions to each frequency as shown below [20].
h i⊤
Definition 4. Xt = sin (f1 ) , cos (f1 ) , . . . , sin f d , cos f d
2 2

The use of a single network
Algorithm 2 R1SM-T Decomposition
that predicts a component for each
similarity matrix based on posi- Require: Time-series of similarity matrices D =
tional embeddings permits informa- [D1 , D2 , ...DT ], early stopping threshold δ, and
tion sharing across time while simul- penalty term λ
taneously allowing the model to ad- 1: function R1SM-T(D, δ, λ)
just the results over time. In doing 2: Y 1 ← D, i ← 0
so, the model gains the ability to 3: do
learn meaningful results during pe- 4: i←i+1
riods in which less data is available, 5: Initialize network F (·)
adapting to the rate of change that 6: while F has not converged do
is present in the data. That is to 7: ℓ ← 0
say, if time is not relevant at all the 8: r i,1 , r i,2 , ...r i,T ← F (X)
model can learn to ignore the input 9: for t ← 1 to T do
embedding Xt entirely. If time is rel- 10: U t ← min(triu(r i,t r ⊤ i,t , 1) - Yi,t , 0)
evant, the embeddings Xt and Xt+∆ 11: O t ← max(Y i,t - triu(r i,t r i,t , 1), 0)
⊤

have a relationship that can be ex- 12: ℓ ← ℓ + ∥λU t + O t ∥2
tracted by a single layer of a neural 13: Back-propagate ℓ and run optimizer step.
network [20], allowing for informa- 14: r i,1 , r i,2 , ...r i,T ← F (X)
tion sharing over time. This infor- 15: for t ← 1 to T do
mation sharing is important as we 16: Ri,t ← min(triu(r i,t r ⊤ i,t , 1), Y i,t )
observe different rates of change over 17: Y ← Y − R
P i+1,t i,t i,t
time, and the amount of samples per
18: while PRDi ≥ δ
month varies by up to three orders
19: return r 1 , r 2 , ...r i−1
of magnitude (as shown in Figure 1).
During each iteration i a new
neural network F (·) optimizes the values in components ri = [r i,1 , r i,2 , ...r i,T ] such that they each
maximally explain their respective matrices in Y i (lines 6 - 13). The loss ℓ of F (X) is computed
using two matrices Ut and Ot , which represent element-wise differences between triu(r i,t r ⊤ i,t , 1)
and Yi,t per timestep. Ut stores under-predictions in triu(r i,t r i,t , 1) (line 10) and Ot stores over-
⊤

predictions in triu(r i,t r ⊤
i,t , 1) (line 11). F (·) is strongly discouraged from over-predicting Yi by
the λ hyper-parameter, which has a value in the range (0, 1], and is set to 0.01 by default. λ
acts as a scaling factor between U and O, causing values in O to contribute more heavily to
the loss (line 12). Due to this term, over-prediction of the values in the components is rare.
Once the batch loss has been computed, the model performs back-propagation and the opti-

Proceedings of the Conference on Applied Machine Learning for Information Security, 2021
R1SM Decomposition For Modeling Antivirus Consensus R. Joyce, E. Raff, and C. Nicholas

mizer step (line 13). Training continues until the model converges, at which point r i holds an
optimal solution. Algorithm 2 can solve the R1SM decomposition of a single similarity matrix
by defining it as a time-series with only one timestep. Our implementation of the neural net-
work F (·) uses ten hidden layers with five residual connections. The default hidden layer size is
1,024 neurons, and the network includes multiple bottleneck layers whose sizes are a function of
the input and output layer sizes. The exp() function is applied to all weights in the output layer
of F (·), constraining the predicted components to be non-negative as required by the definition
of the R1SM decomposition. An important design factor is the use of a very small learning rate,
which allows precise adjustments to the values in the component during the learning process.
By default, R1SM-T uses a learning rate of 1e-7. We note that in extended tests a wide array
of layer depths, residual and/or simple feed-forward connections, and numbers of neurons per
layer, produced qualitatively and quantitatively the same results, as the networks are learning
to predict population level statistics without explicit features about the populations, forcing the
network to learn consistent population behaviors.

5 First-Order Interactions in Detection Percent Agreement Decomposition
r , r , ... r
0 1.0

Antivirus Scan Data 5
10
15
0 20 40 60 80
Now that we have introduced the R1SM de- 0 0 0.8

composition and R1SM-T, we use them to
study first-order interactions amongst the an- 20 20
0.6
tivirus engines in the VirusTotal-VT dataset.
First, we investigate the validity of the indus- 40 40
r , r , ... r

try assumption that consensus between an- 0.4

tivirus engines is caused by first-order inter-
60 60
actions, such as sharing of threat intelligence
0.2
and copying from leading vendors. We iden-
tify clusters of antivirus engines with strong
80 80

first-order interactions. Finally, we research 0 10 0 20 40 60 80
0.0

triu(r r + r r + ... + r r , 1)
how first-order interactions between antivirus
engines have changed over a decade. Figure 4: R1SM decomposition of the detection
agreement similarity matrix for VirusShare-VT. The
leftmost subplot contains the sixteen components
5.1 Applying R1SM to Antivirus and the topmost subplot contains their transposes.
Similarity Matrices The large central subplot contains the portion of the
similarity matrix explained by the components.
Figure 4 displays the R1SM decomposition of
the similarity matrix shown in Figure 2(a), Component 1

which measures pairwise detection agree- AVG Emsisoft K7AntiVirus Sophos

ment amongst the antivirus engines in the AhnLab-V3 F-Prot Kaspersky Symantec
Avast F-Secure McAfee TrendMicro
VirusShare-VT dataset. This decomposition BitDefender Fortinet McAfee-GW-Edition TrendMicro-HouseCall

was obtained by applying Algorithm 2 to the CAT-QuickHeal GData Microsoft VBA32

similarity matrix, represented as a time-series
Comodo Ikarus Panda nProtect
DrWeb Jiangmin Rising

with a single timestep. Using an early stop-
ping threshold of δ = 0.1%, the decomposi- Component 2 Component 4 Component 5

tion yielded k = 16 components which explain Authentium

McAfee+Artemis
ALYac

Arcabit
PandaB3

PandaBeta
60.596% of the matrix. That approximately a-sqared

40% of the matrix went unexplained implies
Figure 5: Clusters extracted from the R1SM decom-
that significant amounts of the consensus be- position of the detection percent agreement matrix
tween antivirus engines cannot be explained (δ = 0.1%, ϵ = 0.85).
by first-order interactions alone, which runs

Proceedings of the Conference on Applied Machine Learning for Information Security, 2021
R1SM Decomposition For Modeling Antivirus Consensus R. Joyce, E. Raff, and C. Nicholas

counter to current belief. Determining the Classification Percent Agreement Decomposition
r , r , ... r
nature of such interactions is a subject for fu- 0 1.0

ture research. 5
10

Figure 5 displays clusters extracted from 15
20
the R1SM decomposition in Figure 4 using ϵ = 0 0
0 20 40 60 80 0.8

0.85. Components with less than two antivirus
engines exceeding ϵ are not shown. The clus- 20 20 0.6
tering illustrates a common trait of the R1SM
decomposition, namely that the first compo-
40 40
nent tends to subsume a large quantity of the

r , r , ... r
0.4

similarity matrix, resulting in a large cluster
for the first component. The cluster extracted 60 60

from the first component indicates that a sig- 0.2

nificant number of first-order interactions ex- 80 80

ist between a large group of antivirus engines. 0.0
0 10 20 0 20 40 60 80
Inspection of the clustering shows pairs of an- triu(r r + r r + ... + r r , 1)
tivirus engines with a shared vendor, such as Figure 6: R1SM decomposition of the antivirus clas-
TrendMicro and TrendMicro-Housecall as well sification agreement similarity matrix.
as PandaB3 and PandaBeta. Other antivirus
engines in the clusters have been previously Component 1 Component 2 Component 5

reported to have similarities, such as Bit- ClamAV Avast5 ALYac

Defender, Emsisoft, and GData; McAfee, Comodo K7AntiVirus Ad-Aware
nProtect MicroWorld-eScan
McAfee-GW-Edition, and Microsoft; and
Avast, AVG, and Fortinet [26, 16]. Further Component 7 Component 9 Component 10
investigation is needed to identify the causes Authentium McAfee Ewido

of the first-order interactions between the re- Command McAfee+Artemis NOD32v2

maining antivirus engines. F-Prot SAVMail

UNA
Figure 6 shows the R1SM decomposi- Component 11 Component 13
tion of the similarity matrix shown in Fig-
K7AntiVirus FortinetBeta
ure 2(b), which contains pairwise classification K7GW McAfeeBeta

agreement scores for the antivirus engines in
VirusShare-VT. This decomposition has k = Figure 7: Clusters extracted from the R1SM decom-
21 components which explain 58.394% of the position of the classification percent agreement ma-
matrix. As with the prior decomposition, trix (δ = 0.1%, ϵ = 0.7).
a significant portion of the similarity matrix
cannot be explained using first-order interactions alone, and further work is necessary to identify
and model the complex relationships between this set of antivirus engines. Comparing the cen-
tral subplots of figures 4 and 6 shows that both decompositions are structurally alike, indicating
that many of the same first-order interactions exist between the antivirus engines whether mea-
suring detection or classification agreement. We revisit this observation in Section 5.2, where
we show that the time-series for the two similarity metrics also have R1SM-T decompositions
with notable similarities.
Figure 7 shows the clusters extracted from the classification percent agreement R1SM de-
composition in Figure 6 using ϵ = 0.7. Again, components with less than two antivirus engines
exceeding ϵ are not displayed. Shared vendor relationships between Authentium and Command,
McAfee and McAfee+Artemis, and K7AntiVirus and K7GW are identified by the clusters for
components 7, 9, and 11 respectively. Zhu et al. [26] identify similarities between ClamAV and
Comodo (component 1) as well as Ad-Aware and MicroWorld-eScan (component 5). Sebastián
et al. [16] also report that the Ad-Aware and MicroWorld-eScan engines frequently have iden-
tical labels. No prior work has identified similarities between any antivirus engines developed
by Fortinet and McAfee, but in 2019 the two vendors released a joint endpoint security solution

Proceedings of the Conference on Applied Machine Learning for Information Security, 2021
R1SM Decomposition For Modeling Antivirus Consensus R. Joyce, E. Raff, and C. Nicholas

[4]. A partnership between Fortinet and McAfee likely accounts for the first-order interactions
between their two beta engines in component 13. We have not found any publicly known con-
nections between the remaining clustered antivirus engines.

5.2 Applying R1SM-T to Time-Series of Antivirus Similarity Matrices
Next, we investigate the changes in first-order interactions between antivirus engines in the
VirusShare-VT dataset over the course of a decade. To do this, we separated VirusShare-VT
into groups of antivirus scans by month, and computed detection and classification agreement
similarity matrices for each group. The similarity matrices were then arranged into two time-
series representing monthly change in classification and detection agreement respectively. Fi-
nally, we applied R1SM-T to both time-series.
The R1SM-T models for the detection and
classification agreement time-series converged
after 5,200,000 and 5,440,000 training itera-
tions respectively. They each identified k = 26
sets of components using the early stopping
value δ = 0.1%. The R1SM-T decomposi-
tion for the detection percent agreement time-
series explains an average of 73.709% of the
matrices and the decomposition for the clas-
sification percent agreement time-series ex-
plains an average of 67.196% of the matri-
ces. Interestingly, the percent explained by
the R1SM-T decomposition varies monthly, as
shown in Figure 8. In this figure, the upper
red line of each plot indicates monthly changes
in synchronicity, originally shown in Figure
3. Each region shaded in blue represents how
much a component of the decomposition con-
tributes
P to the monthly synchronicity, given
Ri,t
by n(n−1)/2 . Synchronicity that cannot be ex- Figure 8: Monthly detection and classification syn-
plained by first-order interactions captured in chronicity explained by R1SM-T. Synchronicity con-
tributed by each component is shown in a shade of
the decomposition are represented by the area
blue, with component 1 at the bottom. The total
shaded in red. In both plots, the proportion of monthly synchronicity is indicated by the topmost
synchronicity explained by first-order interac- red line. The red shaded region indicates how much
tions slowly increases. Although the cause of synchronicity is not explained by first-order interac-
this trend is unknown, a possible explanation tions contained within R1SM-T components.
is an increase in sharing of threat intelligence
throughout the industry over time. In both plots, the first component steadily becomes the dom-
inant contributor to the explained synchronicity over time. Before 2009, the other components
supplied approximately half of the explained synchronicity, but they became negligible by 2014.
This seems to indicate that sharing of threat intelligence used to be limited to disparate groups
of antivirus engines, but over time information sharing has become ubiquitous. This also corre-
lates with usage of VirusTotal itself within industry, as it provides extensive threat intelligence
tooling and a community-based platform for sharing information about malware samples.
Next, we investigate the first R1SM-T component of both time-series due to its intriguing
behavior in Figure 8. In doing so, we observe how the behaviors of individual antivirus engines
as well as overall trends in the antivirus community change over time. Figures 9 and 10 display
the first component of the R1SM decomposition for each of the 121 similarity matrices in the
two time-series. Each column represents the component for a particular month, and each row

Proceedings of the Conference on Applied Machine Learning for Information Security, 2021
R1SM Decomposition For Modeling Antivirus Consensus R. Joyce, E. Raff, and C. Nicholas

indicates how the contributions of a specific antivirus engine to the first component have changed
over time.
The overall magnitude of the components
within Figure 10 is lower than their counter- Detection Time-Series Component 1
parts in Figure 9, and month-to-month com- 0 1.0

ponent values have more variability. How-
ever, the similarity in structure between the 20
0.8

two decompositions is striking. As with our
earlier findings for the two R1SM decompo-
0.6
sitions, a possible explanation for this struc-

Antivirus Engine
40

tural similarity is that classification depends
upon detection. These results could also indi- 60
0.4

cate that the same types of first-order interac-
tions tend to exist between antivirus engines 0.2

regardless of whether detection or classifica- 80

tion agreement is measured. Next, we discuss 0.0
0 20 40 60 80 100 120
notable types of features visible in the decom- Time Step

position that indicate changes in first-order in- Population wide changes in Product entry Sudden drops in Slow change in
correlation never previously correlation correlations
teractions between antivirus engines. identified

Trivial Patterns: Insights into alter- Figure 9: The first components for the time-series of
ations in antivirus behavior can be observed similarity matrices measuring monthly antivirus de-
when corresponding values in the decomposi- tection percent agreement in VirusShare-VT. Each
tion change radically within a short time pe- column is the component for a particular month,
riod. Both decompositions clearly display the starting in May 2006 and ending in May 2016.
months during which antivirus engines were
added to the VirusTotal platform, such as Classification Time-Series Component 1
Alyac in Nov. 2014 (row 0) [22]. The Jun. 0 1.0

2015 retirement of the Norman antivirus en-
gine from VirusTotal is also visible in both de- 0.8
20
compositions (row 56) [23].
Abnormal Structural Changes: Ver-
0.6
tical "bands" in the R1SM-T decompositions
Antivirus Engine

indicate periods of change within the entire
antivirus community that have never been 60
0.4

previously noted or identified, to the best of
our knowledge. A band evident in both figures 0.2

9 and 10 takes place during Apr. and May 80

2011 (columns 59 and 60), in which values for 0.0

a number of antivirus engines, including Avast
0 20 40 60 80 100 120
Time Step

(row 13), Emsisoft (row 30), F-Prot (row 32), Correlation change over time not
present from just detection plots.
Sudden drops in Global structure change
correlation only for classification
GData (row 38), Ikarus (row 39), Rising (row
65), Sophos (row 69), TheHacker (row 74), Figure 10: The first components for the time-series
VIPRE (row 80) drop sharply. A second band measuring antivirus classification percent agreement
beginning in Jul. 2014, which lasts until Feb. in VirusShare-VT.
2015 in Figure 9 and until May 2015 in Figure
10, indicates a turbulent period where the relationships between antivirus engines were in flux.
The components in Figure 10 immediately following this band change drastically, with many an-
tivirus engines gaining an increased share of the component in comparison to the prior months.
To understand the cause of these community-wide disturbances in correlation requires further
research, but should immediately impact how industry design their label aggregation pipelines.
We would recommend any training data labeled during these time periods be regarded as po-

Proceedings of the Conference on Applied Machine Learning for Information Security, 2021
R1SM Decomposition For Modeling Antivirus Consensus R. Joyce, E. Raff, and C. Nicholas

tentially suspect, and such data should undergo further analysis to confirm label quality.
Sudden Variations: Individual changes to an antivirus within a short period of time also
indicate notable events. In Figure 9 a large gap occurs for K7Antivirus (row 41) from Feb. 2010
to Jul. 2010, which corresponds with the release of K7 TotalSecurity version 10.0 on Feb. 23,
2010 [15]. Aegislab (row 4) fluctuates significantly for unknown reasons, dropping from 0.575
when it was first introduced to VirusTotal in Feb. 2014 [21] to 0.146 and rising back to a peak of
0.716 in Aug. 2014. Aegislab’s contributions to the first component are nearly identical to those
of Alibaba (row 7) throughout all of 2015, possibly indicating a common information source.
Differences Between Decompositions: Since the first components of both R1SM-T
decompositions are structurally very similar, differences between the two may indicate first-order
correlations caused by factors related to either benign/malicious detection or family classification
alone. These factors could include increased or reduced use of heuristic antivirus signatures or
changes in malware family naming conventions. External events, such as the emergence of new
malware families, could also explain these discrepancies.
The R1SM-T decompositions in Figures 9 and 10 reveal that correlations between antivirus
engines can change significantly within a short time period. Furthermore, they illustrate periods
of industry-wide change that have never been previously identified. Although we explain many
of the features in the decompositions, the factors that cause consensus between antivirus engines
to change are still largely unknown, and identifying the sources that cause periods of population-
wide volatility is especially important.

6 Discussion
We lack complete understanding of the factors that cause correlations between antivirus engines;
first-order interactions alone are not sufficient for modeling the complex interconnections between
antivirus engines. In studying how consensus amongst antivirus engines change over time, we
found that the relationships between antivirus engines are even more intricate and volatile than
previously thought. The overall level of consensus amongst antivirus engines can change quickly
in short periods of time for reasons which are still not fully understood. Using R1SM-T we
found that first-order interactions have become increasingly responsible for consensus between
antivirus engines over time, although they are still insufficient for modeling some of the sources
of antivirus correlations. Furthermore, we found that first-order interactions now seem to be
nearly ubiquitous across the entire antivirus industry, whereas disparate segments of the industry
previously existed where first-order interactions could not be identified. Finally, we showed that
components of R1SM-T could be utilized to identify individual and population-wide changes in
antivirus behavior.
Current understanding of antivirus dynamics is clearly insufficient and more research about
the causes of antivirus correlation is needed. It is difficult to trust antivirus results when the
factors that cause them to be correlated are still poorly understood. On account of this, and be-
cause relationships between antivirus engines can change significantly in a short period of time,
existing methods for aggregating antivirus signatures for the purposes of malware detection and
classification are flawed. Future aggregation approaches should consider weighted ensembles
where the weights of the voting members are also a function of time. We also hope that ele-
ments of this work, such as the ability to quantify first-order relationships and assess changes
in these relationships over time, may themselves contribute towards improvements in antivirus
aggregation.

Proceedings of the Conference on Applied Machine Learning for Information Security, 2021
R1SM Decomposition For Modeling Antivirus Consensus R. Joyce, E. Raff, and C. Nicholas

Appendix A

Index Antivirus Engine Scan Count Index Antivirus Engine Scan Count
0 ALYac 4,679,821 47 McAfee+Artemis 995,699
1 AVG 24,982,795 48 McAfee-GW-Edition 24,607,621
2 AVware 5,664,526 49 McAfeeBeta 95,784
3 Ad-Aware 12,649,803 50 MicroWorld-eScan 19,382,987
4 AegisLab 10,636,692 51 Microsoft 24,984,940
5 Agnitum 19,009,698 52 NANO-Antivirus 18,763,016
6 AhnLab-V3 23,792,676 53 NOD32 4,738,012
7 Alibaba 4,657,472 54 NOD32Beta 343,198
8 AntiVir 19,225,387 55 NOD32v2 245,233
9 Antivir7 8,710 56 Norman 21,187,821
10 Antiy-AVL 24,559,174 57 PCTools 11,426,342
11 Arcabit 3,770,802 58 Panda 24,713,903
12 Authentium 1,778,326 59 PandaB3 2,695
13 Avast 24,945,279 60 PandaBeta 288,403
14 Avast5 2,405,851 61 PandaBeta2 3,371
15 Avira 5,432,038 62 Prevx 3,826,154
16 Baidu 752,127 63 Prevx1 438,718
17 Baidu-International 13,770,014 64 Qihoo-360 11,703,897
18 BitDefender 25,037,371 65 Rising 24,233,086
19 Bkav 13,155,628 66 SAVMail 149,834
20 ByteHero 20,476,926 67 SUPERAntiSpyware 23,567,247
21 CAT-QuickHeal 25,048,885 68 SecureWeb-Gateway 101,352
22 CMC 11,819,328 69 Sophos 24,540,725
23 ClamAV 25,007,198 70 Sunbelt 1,741,218
24 Command 250,234 71 Symantec 24,732,139
25 Commtouch 17,141,359 72 T3 18,561
26 Comodo 24,666,456 73 Tencent 8,606,167
27 Cyren 5,885,738 74 TheHacker 25,083,441
28 DrWeb 24,599,303 75 TotalDefense 20,294,251
29 ESET-NOD32 19,964,248 76 TrendMicro 24,752,548
30 Emsisoft 22,526,376 77 TrendMicro-HouseCall 23,560,584
31 Ewido 292,281 78 UNA 44,282
32 F-Prot 25,036,409 79 VBA32 24,870,783
33 F-Prot4 26,895 80 VIPRE 23,181,524
34 F-Secure 24,060,788 81 ViRobot 24,843,379
35 FileAdvisor 129,111 82 VirusBuster 5,365,970
36 Fortinet 25,045,656 83 Webwasher-Gateway 202,898
37 FortinetBeta 89,667 84 Yandex 564,996
38 GData 24,831,415 85 Zillya 8,444,657
39 Ikarus 25,087,466 86 Zoner 7,351,648
40 Jiangmin 24,011,547 87 a-squared 1,135,672
41 K7AntiVirus 24,510,017 88 eSafe 9,936,896
42 K7GW 16,303,096 89 eScan 37,174
43 Kaspersky 24,721,716 90 eTrust-InoculateIT 35,112
44 Kingsoft 17,919,699 91 eTrust-Vet 4,731,780
45 Malwarebytes 18,710,973 92 nProtect 24,252,689
46 McAfee 24,949,683
Table 1: Antivirus engines present in at least 1,000 scan reports in VirusTotal-VT. The Index column
displays which row and/or column in Figures 2(a), 2(b), 4, 6, 9, and 10 each engine corresponds to.

Proceedings of the Conference on Applied Machine Learning for Information Security, 2021
R1SM Decomposition For Modeling Antivirus Consensus R. Joyce, E. Raff, and C. Nicholas

References
[1] Virusshare.com - because sharing is caring. https://virusshare.com/, Last accessed on
2020-3-9.

[2] U. Bayer, P. M. Comparetti, C. Hlauschek, C. Kruegel, and E. Kirda. Scalable, behavior-
based malware clustering. In NDSS 2009, 16th Annual Network and Distributed System
Security Symposium, February 8-11, 2009, San Diego, USA, San Diego, UNITED STATES,
02 2009. URL http://www.eurecom.fr/publication/2783.

[3] L. Breiman. Bagging predictors. Machine Learning, 24(2):123–140, 1996. ISSN
08856125. doi: 10.1007/BF00058655. URL http://www.springerlink.com/index/
10.1007/BF00058655.

[4] Fortinet. https://www.fortinet.com/content/dam/fortinet/assets/alliances/sb-
fortinet-mcafee-solution.pdf, Last accessed on 2021-2-17.

[5] I. Gashi, B. Sobesto, V. Stankovic, and M. Cukier. Does malware detection improve with
diverse antivirus products? an empirical study. In F. Bitsch, J. Guiochet, and M. Kaâniche,
editors, Computer Safety, Reliability, and Security, pages 94–105, Berlin, Heidelberg, 2013.
Springer Berlin Heidelberg. ISBN 978-3-642-40793-2.

[6] M. Hurier, K. Allix, T. F. Bissyandé, J. Klein, and Y. Le Traon. On the lack of consensus
in anti-virus decisions: Metrics and insights on building ground truths of android malware.
In J. Caballero, U. Zurutuza, and R. J. Rodríguez, editors, Detection of Intrusions and
Malware, and Vulnerability Assessment, pages 142–162, Cham, 2016. Springer International
Publishing. ISBN 978-3-319-40667-1.

[7] R. A. Jacobs, M. I. Jordan, S. J. Nowlan, and G. E. Hinton. Adaptive Mix-
tures of Local Experts. Neural Computation, 3(1):79–87, feb 1991. ISSN 0899-7667.
doi: 10.1162/neco.1991.3.1.79. URL http://www.mitpressjournals.org/doi/10.1162/
neco.1991.3.1.79.

[8] Y. Jiang, S. Li, and T. Li. Em meets malicious data: A novel method for massive mal-
ware family inference. In Proceedings of the 2020 3rd International Conference on Big
Data Technologies, ICBDT 2020, page 74–79, New York, NY, USA, 2020. Association
for Computing Machinery. ISBN 9781450387859. doi: 10.1145/3422713.3422743. URL
https://doi.org/10.1145/3422713.3422743.

[9] A. Kantchelian, M. C. Tschantz, S. Afroz, B. Miller, V. Shankar, R. Bachwani, A. D. Joseph,
and J. D. Tygar. Better malware ground truth: Techniques for weighting anti-virus vendor
labels. In ACM Workshop on Artificial Intelligence and Security, 2015.

[10] T. Kolda and B. Bader. Tensor decompositions and applications. SIAM Rev., 51:455–500,
2009.

[11] I. Martín, J. A. Hernández, and S. de los Santos. Machine-learning based analysis and
classification of android malware signatures. Future Generation Computer Systems, 97:
295–305, 2019. ISSN 0167-739X. doi: https://doi.org/10.1016/j.future.2019.03.006. URL
https://www.sciencedirect.com/science/article/pii/S0167739X18325159.

[12] A. Mohaisen and O. Alrawi. Av-meter: An evaluation of antivirus scans and labels. In
S. Dietrich, editor, Detection of Intrusions and Malware, and Vulnerability Assessment -
11th International Conference, DIMVA 2014, Egham, UK, July 10-11, 2014. Proceedings,
volume 8550 of Lecture Notes in Computer Science, pages 112–131. Springer, 2014. doi:
10.1007/978-3-319-08509-8\_7. URL https://doi.org/10.1007/978-3-319-08509-8_7.

Proceedings of the Conference on Applied Machine Learning for Information Security, 2021
R1SM Decomposition For Modeling Antivirus Consensus R. Joyce, E. Raff, and C. Nicholas

[13] A. Moser, C. Kruegel, and E. Kirda. Limits of static analysis for malware detection. In
Twenty-Third Annual Computer Security Applications Conference (ACSAC 2007), pages
421–430, 2007. doi: 10.1109/ACSAC.2007.21.

[14] E. Raff and C. Nicholas. A Survey of Machine Learning Methods and Challenges for
Windows Malware Classification. In NeurIPS 2020 Workshop: ML Retrospectives, Surveys
& Meta-Analyses (ML-RSA), 2020. URL http://arxiv.org/abs/2006.09271.

[15] D. Salt. http://www.articletrader.com/computers/software/k7-computing-
launches-new-customer-awareness-and-brand-campaign-for-k7-total-security-
10.html, Last accessed on 2021-2-18.

[16] M. Sebastián, R. Rivera, P. Kotzias, and J. Caballero. https://github.com/malicialab/
avclass/blob/master/avclass/lib/avclass_common.py, Last accessed on 2021-2-16.

[17] M. Sebastián, R. Rivera, P. Kotzias, and J. Caballero. Avclass: A tool for massive malware
labeling. In F. Monrose, M. Dacier, G. Blanc, and J. Garcia-Alfaro, editors, Research
in Attacks, Intrusions, and Defenses, pages 230–253, Cham, 2016. Springer International
Publishing. ISBN 978-3-319-45719-2.

[18] J. Seymour. Labeling the virusshare corpus- lessons learned. BSides Las Vegas.
https://www.peerlyst.com/posts/bsideslv-2016-labeling-the-virusshare-corpus-
lessons-learned-john-seymour, Last accessed on 2020-11-20, 2016.

[19] E. C. Spafford. Is anti-virus really dead? Computers & Security, 44:iv, 2014.
ISSN 0167-4048. doi: https://doi.org/10.1016/S0167-4048(14)00082-0. URL http://
www.sciencedirect.com/science/article/pii/S0167404814000820.

[20] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. u.
Kaiser, and I. Polosukhin. Attention is all you need. In I. Guyon, U. V. Luxburg,
S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, Ad-
vances in Neural Information Processing Systems, volume 30, pages 5998–6008. Cur-
ran Associates, Inc., 2017. URL https://proceedings.neurips.cc/paper/2017/file/
3f5ee243547dee91fbd053c1c4a845aa-Paper.pdf.

[21] VirusTotal, . https://blog.virustotal.com/2014/02/virustotal-aegislab.html, Last
accessed on 2021-2-18.

[22] VirusTotal, . https://blog.virustotal.com/2014/11/virustotal-alyac.html, Last ac-
cessed on 2021-2-12.

[23] VirusTotal, . https://blog.virustotal.com/2015/06/virustotal-norman.html, Last ac-
cessed on 2021-2-12.

[24] D. Votipka, S. Rabin, K. Micinski, J. S. Foster, and M. L. Mazurek. An Observational
Investigation of Reverse Engineers’ Process and Mental Models. In Extended Abstracts of
the 2019 CHI Conference on Human Factors in Computing Systems, 2019. doi: 10.1145/
3290607.3313040.

[25] D. H. Wolpert. Stacked generalization. Neural networks, 5:241–259, 1992. URL http:
//www.sciencedirect.com/science/article/pii/S0893608005800231.

[26] S. Zhu, J. Shi, L. Yang, B. Qin, Z. Zhang, L. Song, and G. Wang. Measuring and modeling
the label dynamics of online anti-malware engines. In 29th USENIX Security Symposium
(USENIX Security 20), Boston, MA, Aug. 2020. USENIX Association. URL https://
www.usenix.org/conference/usenixsecurity20/presentation/zhu.

Proceedings of the Conference on Applied Machine Learning for Information Security, 2021