<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>Heated Alert Triage (HeAT): Network-Agnostic Extraction of Cyber Attack Campaigns</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Stephen Moskal</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Shanchieh Jay Yang</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>Rochester Institute of Technology</institution>
        </aff>
      </contrib-group>
      <pub-date>
        <year>2021</year>
      </pub-date>
      <abstract>
        <p>With growing sophistication and volume of cyber attacks combined with complex network structures, it is becoming extremely dificult for security analysts to corroborate evidences to identify campaigns and threats on their network. So much so that organizations employ teams of security professionals just to keep up with vast amount of data presented to the analysts each day. This work develops HeAT (Heated Alert Triage): given a critical indicator of compromise (IoC) such as a severe IDS alert, HeAT produces a HeATed Attack Campaign depicting the actions that led up to the critical event including reconnaissance and initial exploitation stages. We define the concept of “Alert Episode Heat" to represent the analysts opinion of how much an event contributes to the attack campaign of the critical IoC given their own knowledge of their network context and security expertise. Leveraging a network-agnostic feature set and a short but targeted training process, HeAT is able to realize insightful and concise attack campaigns for IoC's not observed before, compare attack strategies of diferent attackers with the same IoC, and also be applied across networks with the same degree of fidelity. HeAT maintains the analysts original assessment of the specified “HeAT" regardless of the critical event being assessed or the network topology. We demonstrate the capabilities of HeAT with case studies using cyber-competition datasets to mimic how HeAT would be deployed in practice and assess the HeATed attack campaign from the analyst's perspective. With the goal of aiding the analyst in quickly finding further evidence of an attack, we show that HeAT immediately reveals each attack stage of an attack campaign embedded deeply within millions of alerts that may have needed a whole team of analysts to achieve otherwise.</p>
      </abstract>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>Introduction</title>
      <p>Threats of sophisticated and highly impactful cyber attacks have become so common that many
organizations have implemented “Security Operations Centers" (SOC) to investigate, respond
to, and hunt potential threats within networks. SOC’s typically implement a tiered structure
where a tier 1 analyst triages the network for critical events which may be escalated to a tier
2 analyst who will respond to the incident. Assume the role of a tier 1 SOC analyst and you
observe a critical alert, “ GPL EXPLOIT CodeRed v2 root.exe access ", targeting a customer
database. While occurring on a critical asset, a single alert may not be enough evidence to
escalate to the tier 2 analyst and you must now look for other “Indicators of Compromise" (IoC)
to develop more evidence that the alert was indeed caused by an adversary. This is known as
a “triage" and is typically a time consuming and mostly manual process sometimes involving
multiple analysts to comb through lengthy log files to find other IoC’s related to the initial IoC.
With the inflation of network sizes and the general increase of foreign threats broadly targeting
any type of organization, many SOC analysts are overwhelmed with the amount of log data from</p>
      <p>Intrusion Detection Systems (IDS) which hampers their ability to quickly assess their network
for threats.</p>
      <p>Given a critical alert (an IoC) and IDS alert logs, we ask if we can leverage machine learning
techniques to aid the analyst in the triage process and automatically reveal other steps the
adversary took to “arrive" at their goal. The compilation of the actions detailing each “stage"
of the attack is called an “attack campaign" which would describe how, when, and where the
attacker learned about the network, gained initial access, and then eventually achieving their
goal. Developing this attack campaign from IDS alert logs can be extremely dificult as the
analyst must consider for each alert: the network context, related attributes between the alerts,
and their own expertise to determine the relationship between the critical alert and prior alerts.
These considerations sometimes leads to subjectivity of the actual contribution of the alert to
the attack campaign. We envision an automated triage system to reflect the analyst’s opinion
on the types of events that they believe are a part of an attack campaign and ability to apply
that “thinking" to other triages in the future.</p>
      <p>We propose a system, HeATed Alert Triage (HeAT), to perform automated triaging of IDS
alerts. Given a critical IDS alert, HeAT creates a “HeATed Attack Campaign" (HAC) using a set
of network agnostic features and a small set of analyst defined critical alert episode relations. In
the form of aggregated alerts defined as “Alert Episodes," the HAC’s generated by HeAT tells the
story of the attacker’s progression leading to a critical event. HeAT estimates the "Alert Episode
HeAT” (AEH) for each alert episode with respect to the critical alert to describe the episodes
contribution to the attack campaign given how the analyst has interpreted AEH previously.
We have developed HeAT with reusability and transferability in mind; we use network agnostic
features so that HeAT can uncover attack campaigns for other critical alerts, adversaries, or
networks. We envision HeAT to be used by SOC analysts to display the HAC once they observe
the first IoC so that they can quickly determine if further action is needed for not only one attack
type but for many. Note that we demonstrate the methodology and capability of HeAT with
one specific IDS, Suricata, in this work, while the network agnostic features are generalizable to
treat heterogeneous alerts and event logs.</p>
      <p>Using a set of targeted case studies by processing data collected through cyber-competitions,
we demonstrate, in close to real “deploy-able" scenarios, HeAT’s ability to:
1. Leverage a small amount of self-labeled data to discover meaningful insights into attack
campaigns for critical alerts,
2. compare attack strategies for the same critical event and quickly determine key milestones
such as discovery, initial access, and other events leading up to the critical event, and
3. identify attack campaigns under diferent network settings, and reveals non-coincidental
patterns in attack strategies across networks.
2</p>
    </sec>
    <sec id="sec-2">
      <title>Related Work</title>
      <p>
        Extraction and assessments of attack campaigns has been studied in depth in the form of
Attack Graphs (AG) and have the capability to provide detailed insights into how attackers can
traverse a network. AG’s use network topology and vulnerability assessments to define
potential paths through a network an adversary can exploit. AG works employ techniques such as
alert correlation [
        <xref ref-type="bibr" rid="ref18 ref20 ref21 ref23">20, 25, 23, 22</xref>
        ], process-mining [
        <xref ref-type="bibr" rid="ref3 ref5">5, 3</xref>
        ], and Markovian-based approaches [
        <xref ref-type="bibr" rid="ref7 ref8">7, 8</xref>
        ] to
map observables to pre-existing AG’s. However these approaches require a significant amount
of expert knowledge to configure, create attacker scenario templates, and assumes that each
vulnerability is known [
        <xref ref-type="bibr" rid="ref2">2</xref>
        ]. If we constrain our research to find approaches that give AG-like
insight without intimate knowledge of the network and vulnerabilities, we find significantly less
works within academia. Navarro et al. presents HuMa [
        <xref ref-type="bibr" rid="ref14">16</xref>
        ] and OMMA [
        <xref ref-type="bibr" rid="ref15">17</xref>
        ] to extract context
from logs, vulnerability databases like CVE and CAPEC, and analyist feedback to find malware
behaviors. Moskal et al. used a sufix-based Markov chain to derive sequences of aggregated
alerts based on their alert characteristics called attack episodes so that sequences of episodes
could be compared [14]. Landauer et al. [
        <xref ref-type="bibr" rid="ref11">11</xref>
        ] extracts from cyber threat intelligence (CTI)
reports and applies the knowledge to raw log data to report actionable multi-stage scenarios.
Lastly, Nadeem et al. [
        <xref ref-type="bibr" rid="ref13">15</xref>
        ] present SAGE which employs S-PDFA to extract meaningful AG’s
from only intrusion alerts and without prior expert knowledge. However, an issue that plagues
these works is the lack of high quality labeled attack scenario data to comprehensively assess,
compare, and validate the identified attack strategies.
      </p>
      <p>
        In the private sector, where data is more abundant, the concept of AI-driven products to
assess and automatically triaging a network is an extremely fast growing sector. As of 2021,
the adoption of AI/ML techniques to solve cyber security problems has exploded. To name a
few, companies such as DarkTrace with their “Cyber AI Analyst" [
        <xref ref-type="bibr" rid="ref4">4</xref>
        ], IBM with QRadar Advisor
with Watson [
        <xref ref-type="bibr" rid="ref10">10</xref>
        ], and Centripetal with AI-Analyst [
        <xref ref-type="bibr" rid="ref16">18</xref>
        ] all advertise their capabilities to leverage
AI specifically to aid analysts in the triaging process. While these products are undoubtedly
extremely sophisticated due to their substantial resources, it is impossible to assess their true
capabilities due to the proprietary nature of the method and data. While we do not claim to
compete with these products, their existence shows that this is a developing and more notably
a valuable problem to address and document.
3
      </p>
    </sec>
    <sec id="sec-3">
      <title>HeAT: Extraction of a Heated Attack Campaign (HAC)</title>
      <p>Given the IDS alert logs from a network and an IoC such as a critical IDS alert, our objective is
to develop of sequence of alerts likely to be related to the IoC, forming the attack campaign of the
adversary. We define an “Attack Campaign" as the collection of actions in time which describe
each stage of an attack conducted by an adversary leading to some objective. As there will be
no ground truth describing the real attack campaign, we rely on an initial triage to establish
characteristics of an actual attack campaign first. Then we address other technical challenges
such as high alert volume, network-specific attack characteristics, and limited analyst
datalabeling resources to extract meaningful and concise attack campaigns quickly. The summary of
the methods used in HeAT are described below and the system overview is shown in Figure 1.
• Introduce the labeling approach called “Alert Episode Heat" (AEH) as a numeric ranking
system (0-3) representing key milestones of an attack campaign leading towards an IoC,
• propose a short and eficient labeling process to capture the analyst’s reflection of
meaningful relationships between alert episodes contributing to the same attack campaign,
• use an attack stage-based Gaussian smoothing approach to alert aggregation to create
alert episodes indicative of actions performed by adversaries,
• use alert episodes to derive network agnostic features relating characteristics between
episodes, enabling prediction of AEH regardless of attack type or network configuration,
• use ML/AI to learn and predict AEH values for prior episodes to a critical episode, and
• construct and visualize HeATed Attack Campaigns (HAC) with “HeATed episodes".</p>
      <p>In the subsequent sections in this work we define the concept of AEH, describe our process
to aggregate alerts as alert episodes, define our AEH labelling process, and then finally describe
our application of AEH to generate the HAC for a given critical alert.
The concept of Alert Episode Heat (AEH) is a numeric ranking system (0-3) which given a
critical alert episode ec and a prior episode ep, AEH ranks the contribution of ep to the attack
campaign of ec. We use the concept of “alert episodes" to represent groups of alerts that are
indicative of action(s) with a specific impact. Each alert episode may contain one or many
alerts sharing similar attributes, such as attack impact, which may or may not be related to
the campaign of ec. AEH is intended to capture the attacker’s progression towards ec given the
alerts of ep.</p>
      <p>While many IDS’s already have some notion of severity embedded within the alert (Suricata’s
severity attribute), these are typically static and independent from all other alerts that have
occurred. IDS’s such as Suricata have no notion of correlated alerts but simply report suspicious
behavior based on signature matches of known adversarial actions and additional information is
needed to determine if two events are correlated. Additional factors such as the network topology,
the assets contained on specific machines, and the analyst’s own expertise is considered when
correlating the true severity between security events. The concept of Alert Episode Heat (AEH)
is to create these correlations between a critical episode and the episodes prior. Given ec and ep,
our objective is to define an AEH Generator as: h(ep|ec) = f (ep, ec) where {h ∈ R|0 ≤ h ≤ 3}.</p>
      <p>
        We design the AEH values as a small set of discrete values that signify key milestones within
an attack campaign. Table 1 describes the characteristics of the “HeAT levels" which are used
to label and create the initial AEH training set. We use the high-level attack stages such as
“reconnaissance", “exploitation", and “actions on objective" [
        <xref ref-type="bibr" rid="ref12">12</xref>
        ] to represent heat levels 1, 2
and 3, respectively, to reflect their progressive impact on a network. We choose a “less-is-more"
approach as we embed specific attack stage information within our labels and human studies show
that 3 to 4 options is optimal reduce error for human surveys [
        <xref ref-type="bibr" rid="ref1">1</xref>
        ]. With HeAT level representing
a small number of mutually exclusive attack stages, we believe the analyst can quickly determine
an appropriate HeAT level and we believe there will be less ambiguity between HeAT levels.
      </p>
      <p>Description
No relation to critical event
Recon. actions that may provide info. about ec
Exploitation of assets giving access required to achieve ec</p>
      <p>Exfiltration/DoS/Access to info. directly relevant to ec</p>
      <p>Given our focus on episodes we now describe our process for converting individual alert
streams into aggregated alerts to “Alert Episodes" and then features that describe our episodes.
3.2</p>
      <sec id="sec-3-1">
        <title>Alert Episodes with the Action-Intent Framework (AIF)</title>
        <p>When discussing IDS’s it is common knowledge that they are plagued with generating a high
volume of alerts due to false positives, vague signatures, or actions that cause excessively repeated
alerts. Alert aggregation is used to group alerts of similar attributes such as time proximity or
impact to reduce the number of events presented to the analyst but also could represent an action
performed by the adversary. Our main limitation is that we only have the attributes defined
within the Suricata IDS alerts, which is limited in scope, however we use the alert signature
description to deduce the “type" of action the adversary could be performing. Given the alert
attributes, we define an “Alert Episode" to be the set of aggregated alerts for a single source IP
and same attack stage across multiple target IP’s within a similar time proximity. We accept
that source IP is not a totally reliable attribute, we believe it is the best opportunity to capture
alerts caused by one adversary.</p>
        <p>We adopt the Gaussian Smoothing approach as described by Moskal et al. to aggregate alerts
based on source IP, attack stage, and time [14]. Moskal et al. [14] describes a process where
alerts are aggregated based on the fluctuations of alert volume within a time window for specific
IP addresses and Suricata categories to uncover common sub-sequences of attack patterns. We
choose this process due to its efective application of Gaussian smoothing to represent aggregate
alerts whose the alert arrival time may be inconsistent, sporadic, or periodic. Moskal et al.
concludes [14] mentioning that the Suricata alert “category" is a weak representation attack
stages of an well established attack stage framework such as MITRE ATT&amp;CK. The
ActionIntent Framework (AIF), also defined by Moskal et al. [13], was created as an IDS-focused
version of MITRE ATT&amp;CK and other kill chain frameworks. A mapping between Suricata
signatures and a Action-Intent Stage (AIS) is given. Gaussian low-pass filtering is applied to
histograms in time of alert volume for single IP and AIS, where the LPF filter parameter is set
based on the expected duration of the action on a per AIS basis. Certain types of attacks may
have longer duration than others and thus diferent filter sizes are used.</p>
        <p>Our Alert Episodes are derived by evaluating each peak of the AIS-based filtered histograms
and the collection alert(s) contained in-between the two local minima of the corresponding peak
make the episode. An example of this process can be seen in Figure 2. Conducting this process
over each attack stage for each source IP, combining the derived episodes, and sorting the by
the peak episode time gives an abbreviated view of the sequence of “actions" performed by that
adversary.</p>
        <p>This episode representation not only summarizes the alerts but also enables us to define
network-agnostic features to compare episodes. Next, we define our network-agnostic features
used to represent the relationship between two alert episodes and the AEH.
3.3</p>
      </sec>
      <sec id="sec-3-2">
        <title>Network Agnostic Features between Alert Episodes</title>
        <p>We engineer our features with two elements in mind: 1) the features describe relations between
two episodes so that the AEH value can be determined with respect to a critical episode and
2) the features are network agnostic so that the model does not learn network specific HeAT
relations that cannot be applied to other attack types or network configurations. As the episodes
contain set of alerts with a wide variety of complex data types such as IP addresses, alert
signatures, etc., we manually define a set of episode features to represent each of these data
types. Each alert episode contains the attributes shown in Table 2 which are derived from the
alerts contained within the episode.</p>
        <p>We define three types of features to capture diferent aspects of common characteristics
between episodes: 1) Time, 2) IP, and 3) Action based features, shown in Table 3. The
timebased features capture the diferences between the critical alert episode and the prior episodes.
Our IP-based features compare if there are similarities between IP addresses of the two episodes
without defining any details of the IP addresses themselves. Lastly, our action-based features
capture similarities between attack stages, signatures, and port numbers to determine if the two
episodes have a similar network impact.</p>
        <p>Our hypothesis is that these network agnostic features will allow us to uncover a variety
of attack campaigns without detailed network topology or system vulnerabilities. We propose
that these network agnostic features can be used to predict the AEH for other attack types and
be applied to other networks. In the next section we describe our methodology for creating
the AEH Generator and how we leverage a small amount of labeled AEH values to determine
HeATed Attack Campaigns (HAC).
3.4</p>
      </sec>
      <sec id="sec-3-3">
        <title>Alert Episode Heat Generator</title>
        <p>When defining the concept of the AEH generator significant challenges arise as labeled data
describing an attack campaign with respect to IDS alerts generated is few and far between.
Data sets that do exist within the research community are typically either outdated (irrelevant
attack types), unlabeled, and/or represented in a diferent domain ( i.e., packet captures) than
IDS alerts. Instead we have the user conduct an initial “triage" of their IDS alerts, label AEH
values to episodes related to a known IoC, and then use the network-agnostic features to create
a predictive model to “generate" heat given other IoC’s.
3.4.1</p>
        <p>
          HeATing Episodes to Develop HAC
Given an AEH-labeled data set, we train a AEH predictive model known as the AEH generator
as h(ep|ec) = f (ep, ec) where {h ∈ R|0 ≤ h ≤ 3}. We define a HAC for given a critical episode
ec as the set of all prior episodes ep ∈ E where the AEH generator applies a non-zero AEH.
Our requirements for a selecting a machine learning model for this application are bound by our
non-linear features and that the AEH value must be a continuous value. HeAT is implemented
in Python and our AEH generator leverages Fast.AI’s Tabular learner [
          <xref ref-type="bibr" rid="ref6">6</xref>
          ] to predict the AEH.
All features within our data are standardized to have a zero mean and unit variance and we
report our 5-fold cross-validated mean squared error (MSE) for the training data.
        </p>
        <p>The process of extracting the HAC from the our data is similar to our training process where
a critical alert is given by the user, HeAT finds the corresponding episode containing the critical
alert, and then the AEH generator is used to “HeAT" all prior episodes with respect to the
critical episode. We apply HeAT to all prior episodes to give our model the opportunity to
discover episodes that may have significantly contributed to the attack campaign that may not
immediately obvious. The set of HeATed episodes with a non-zero AEH are then considered to
be apart of the HeATed attack campaign of the critical alert. As the episodes may contain many
alerts, we foresee the generator finding small relations to the critical episode and apply a small
amount of HeAT to episodes that may not contribute much to the overall campaign; a minimum
AEH threshold can be applied if the user desires. However, we expect truly impactful episodes
to have significantly higher AEH levels than those with just a few similarities between features.
4</p>
      </sec>
    </sec>
    <sec id="sec-4">
      <title>Datasets for Training and Testing</title>
      <p>
        To demonstrate HeAT’s ability to uncover insightful attack campaigns within IDS alerts, we
choose to use data that is known to have actual examples adversarial behavior. In this work
we use publicly available data from penetration competitions such as CPTC ‘18 [
        <xref ref-type="bibr" rid="ref19">21</xref>
        ] and CCDC
’18 [
        <xref ref-type="bibr" rid="ref9">9</xref>
        ] as it contains the impacts of multiple diferent adversaries in a controlled and isolated
network scenario. Competition sets like this give us the opportunity to use HeAT to discover
minute diferences between attacker strategies with the same critical alert. We set-aside the
alerts of one of the 10 teams as “team train" to create our initial triage data and compare the
results of HeAT against the remaining teams as "team test." The Suricata IDS alerts from the
CPTC ’18 event are publicly accessible from [
        <xref ref-type="bibr" rid="ref17">19</xref>
        ] and we leverage the Suricata-to-AIS mapping
provided by Moskal et al. [13] to gain a better representation the attack stage of each alert
than what is provided from only Suricata. CCDC data is only available as packet captures from
[
        <xref ref-type="bibr" rid="ref22">24</xref>
        ] and we convert the PCAP into Suricata alerts using the PCAP mode of Suricata using
the default Suricata rule set. Table 4 summarizes the characteristics of each of these datasets
demonstrating the diference in the number of sources and targets, the large disparity in alert
volume, and reduction of alerts when represented as alert episodes.
      </p>
      <p>To create the training data for the AEH generator, we ask the user to perform a short initial
triage of the prior episodes with respect to a critical IoC and apply AEH values representing
their opinion of the attackers progress contributing towards the IoC. For each prior episode to
the IoC, we compare attributes between the two episodes such as: time diference, IP address
similarities , AIS, and signatures to derive an appropriate AEH given our experience of the
network, data characteristics, and our own security knowledge. In practice we recognize that
analysts could be faced with an excessive amount of data to be labeled. However in this case, we
assume our network agnostic feature set can lessen the reliance on massive amounts of labeled
data typically needed for machine learning applications along with our intuitive AEH definition
to create a more eficient labeling process.</p>
      <p>To test this assumption, we use the CPTC team set aside, “Team Train", and use those
episodes to create our initial AEH Generator training data. We selected three critical IoC’s to
initially manually triage: 1) “ ETPRO ATTACK_RESPONSE MongoDB Database Enumeration
Request ", 2) “ET EXPLOIT Possible ETERNALBLUE MS17-010 Heap Spray ", and 3) “GPL
EXPLOIT CodeRed v2 root.exe access." These signatures describe data exfiltration, arbitrary
code execution, and root privilege escalation actions respectively and can lead to significant
access and impact if successful. These were selected for the purposes of our intended case
studies where: CodeRed was observed in both CPTC and CCDC; Mongo is exclusive to CPTC;
and Eternal Blue is exclusive to Team Train. We then manually apply AEH to prior episodes
to the critical IoC’s as shown in Table 5 with the distribution of AEH values given in Table 6.</p>
      <p>This training data relating two episodes together through the AEH value given is then
converted into our feature space and used to create the AEH Generator to be used to reveal
future attack campaigns with a small portion of the original alert set. As we show in the next
section, our HeAT process is able to extract key characteristics of the attack campaigns given
the limited amount of labeled data. While we show that the capabilities with just this amount
of data is impressive, we plan on conducting another study in the future with multiple analysts
and across other scenarios to assess how the quality of the initial triage afects the resulting
HAC.
5</p>
    </sec>
    <sec id="sec-5">
      <title>HeATed Attack Campaign (HAC) Analysis Using AEH</title>
      <p>Demonstrating the efectiveness of a process like HeAT is extremely challenging due to the lack
of standardized training sets. Instead we take an alternate approach by performing a set of
case studies that mimics the type of analysis we would expect to see if HeAT was deployed in
a real world scenario. First we perform standard ML techniques to assess our performance of
predicting AEH values given the training data described prior. Performing 5-fold cross validation
and using mean-squared error as our performance metric, we report average MSE across the 5
folds as .175 with a maximum error of .335. This is promising as it seems that our model can
capture the AEH values of our training data appropriately and can be applied to other IoC’s.</p>
      <p>Leveraging this model to predict the HeAT of other attack campaigns, we apply HeAT to the
remaining CPTC teams, “Team Test", using Mongo and CodeRed as our IoC’s. We measure our
reduction of relevant events by comparing the number of HeATed episodes to the total number
of “related" alerts and episodes. A related alert (or episode) in this context is any alert that
shares an IP address (source or destination) with the critical IoC. For the two IoC’s there was
16 HAC’s generated and the summary of those HAC’s are given in Table 7.</p>
      <p>From the perspective of a SOC analyst, this 88% reduction of events needed to be triaged
may lead to more timely response to threats as less time will be spent going through irrelevant
alerts. However this will only be true if the HeATed episodes realized by HeAT are an accurate
representation of the attack campaign conducted by the adversary. We now conduct a set of
case studies to assess the fidelity of the HAC’s generated by HeAT.
5.1</p>
      <sec id="sec-5-1">
        <title>Case Study: Tactics towards CodeRed</title>
        <p>Our CPTC dataset gives us the unique opportunity to investigate diferences in attack strategies
of diferent adversaries given the same objective, or in this case the same critical alert. We
propose the scenario where a user has identified this critical alert in the past, gone through the
HeAT training process for this critical alert, and then re-observes the same critical alert in the
future. Although the impact of the vulnerability is the same, we expect the approach taken
could be diferent and thus a diferent mitigation strategy would be required. We use apply
HeAT to diferent adversaries to demonstrate how HeAT is used to gain additional insight into
attack strategies. We find through this case study that HeAT can reveal attacker strategies of
adversarial behavior unknown to HeAT.</p>
        <p>For this case study, we used the alert “ GPL EXPLOIT CodeRed v2 root.exe access " as it was
prevalent across many adversaries and it leads to a significant amount of access if successful.
To demonstrate the diversity of the HAC’s that can be extracted, we begin by comparing two
particularly interesting HAC’s who’s approaches were vastly diferent from each other. Figure
3a shows the resulting HAC for a “calculated" adversary, where we have discovered four key
steps that leads to the CodeRed signature out of a total of 19 HeATed episodes. Whereas the
HAC for a “script kiddie", shown in Figure 3b, demonstrates a significantly diferent approach
where we show a HAC consisting of 144 episodes to achieve the same objective. Due to the
complexities of the script kiddie HAC, our annotations are as follows: 1) Rapid POP3 &amp; IMAP
attempts, 2) CVE-2014-6271 ’Shellshock’, 3) CodeRed First Attempt, and 4) CodeRed on other
targets, same source. The HeATed episodes of the calculated and script kiddie represents 1458
and 29,015 individual alerts respectively, thus our HAC’s are a substantial reduction of events
an analyst needs to assess.</p>
        <p>(a) “Calculated" HAC
(b) “Script Kiddie" HAC</p>
        <p>Our rationale for distinguishing the “calculated" adversary versus the “script kiddie" is from
our analysis of the HeATed episodes. The “calculated" episodes had a single target
throughout this HAC and significant time between episodes. Whereas the script kiddie maintained a
consistent time between actions and targeted many diferent IP’s with the same action which is
indicative of a scripted process. Without HeAT the count of all episodes occurring within these
time-frames for the calculated and script kiddie adversary was 198 and 498 episodes, HeAT
provides a reduction of the number of episodes to be considered by 92% and 71% respectively.
By investigating the alerts within the smaller amount of HeATed episodes, we were able to gain
insight into the individual campaigns as shown by our annotations.</p>
        <p>Due to our alert episodes representing groups of alerts occurring within a similar time frame
and AIS, we find that interfacing with the individual alerts to be much more manageable
compared to the raw alerts. Here we found indications that vulnerabilities within a mail server is
likely be targeted when CodeRed is seen as both of our adversaries have early episodes targeting
POP and IMAP. By looking further into the HAC we see that the mail server maybe initial access
and both adversaries used other actions such as the ColdFusion vulnerability and the Shellshock
attack to further position themselves within the targeted asset. While other evidence is needed
to definitively prove these observations, we believe the resultant HAC’s provides substantial
evidence for an SOC analyst to escalate investigations to the next tier of analysis. Given that
a network may produce seemingly endless volume of alerts, the realization of attack campaigns
becomes overwhelming to conduct manually and we believe HeAT alleviates the strain so that
prevention measures acted on quicker.
5.2</p>
      </sec>
      <sec id="sec-5-2">
        <title>Case Study: HeATing a Diferent Network</title>
        <p>We now ask, can HeATed episodes from one network explain an attack campaign on another
network? For this case study we use the CCDC dataset, which was conducted over a shorter
time period, contains orders of magnitude more alerts, and we have little to no knowledge about
the real network topology or the adversaries themselves. We find this to be a realistic use-case
where a user may use HeAT from another network to gain insight into new network that they
may be unfamiliar with. Using the exact same AEH generator, CodeRed as our critical alert,
and directly applying HeAT to the CCDC dataset with no extra configuration or training, we
present the HAC for this case study in Figure 4</p>
        <p>While visually the HAC’s of the CPTC and CCDC do not seem to be similar, upon deeper
inspection into the alerts contained within the episodes, we do find some striking similarities
between the HAC’s. Initially, we see step 1 making the initial discovery of the critical asset and
step 2 describes a Peer-to-Peer connection made to a SMB network share. The corresponding
signatures within this episode report that data was transferred to and from this asset. This
is a substantial discovery as it indicates that the SMB share is vulnerable which enabled the
adversary to deliver malware as shown in Step 3. Steps 3-5 in Figure 4 are interestingly similar
episodes that we have seen previously including references to POP3 &amp; IMAP brute force,
Shellshock, and the SMTP verification of root access. Without HeAT we believe that finding such
campaign similarities would be much more dificult as the CCDC data contained two orders of
magnitude more alerts. However our HAC does not tell the whole store and further research
would be needed to determine if these alerts were caused by similar tools used by the adversaries
or purely coincidental.</p>
        <p>The capabilities of HeAT to reveal convincing attack campaigns across multiple networks
seems to be impressive and we demonstrate that in some cases such as CodeRed the campaigns
are similar regardless of the network. These observations indicate that prior observations of
attack campaigns can be generalized using HeAT’s network-agnostic features and can be used to
ifnd similar strategies in other networks. This sort of capability could provide immense value to
analysts hunting threats as HeAT could be applied regardless of the network context while still
providing high quality automated triages. In our future work, we investigate the validity of this
observation through a comprehensive study of many more critical events across more networks.
Given our observations with our case studies however, we have confidence in HeAT’s ability to
provide users the first round of insight into attack campaigns occurring in their network with
simply a single observed alert and a straightforward training process.
6</p>
      </sec>
    </sec>
    <sec id="sec-6">
      <title>Limitations</title>
      <p>Throughout our research of HeAT we have identified several limitations, some technical and
others are methodological limitations. First for technical limitations, we only evaluate HeAT
as a post-mortem analysis tool and do not initially intend for HeAT to be used in real-time
applications. Also the HeAT process only considers network-based IDS Suricata alerts which
does not have the ability to resolve actions that do not produce network trafic nor does it
handle encrypted trafic well. For extended detail in HAC’s we would consider the addition of
other security monitors such as the host-based IDS Zeek or ther vulnerability scanner Nessus to
further enhance HeAT.</p>
      <p>Our methodological limitations stems from our training process: i.e., the data used, the
quality of the analyst’s AEH assessments, and the verification of the produced HAC’s. HeAT
relies on the analyst to provide accurate AEH values but you may ask: 1) What if the AEH
values given are incorrect?, 2) What if you have multiple analysts?, or 3) What if the data is a
bad representation of attack campaigns? Each of these questions could have detrimental impact
to the utility of HeAT, especially in a high-stakes field such as network security. We propose to
conduct human studies of diferent analysts under controlled scenarios to answer these questions.
A future iteration of this work will conduct significantly more case studies of attacker behaviors,
provide more statistical analysis of the completeness/utilities of the HAC’s, and be applied to
a real SOC operation to determine the true real-world capabilities of HeAT. In this case we
have the luxury of knowing our data well and certainly is abundant with adversarial behavior,
something that cannot always be said about real-world data.
7</p>
    </sec>
    <sec id="sec-7">
      <title>Conclusion</title>
      <p>The case studies presented in this paper demonstrated how an analyst could use HeAT to
identify attack campaigns leading to a critical IoC with minimal expert inputs. Our HeATed
Attack Campaigns represents the attack stages in time related to a given critical IoC as set
of Alert Episodes with associated Action-Intent Stage. Defined as the “Alert Episode Heat"
(AEH), the AEH captured the analysts reflection of an episode’s contribution to a specified
attack campaign. We found that our implementation of HeAT with network agnostic feature
descriptions was able to extract attack campaigns from alerts novel to HeAT, show the diference
between diverse attacker strategies, and be applied to multiple networks to perform a HeATed
triage. We demonstrated through analysis of our cyber-competition datasets that HeAT can
quickly show the diference between attacker strategies with the same objective and do so in
a concise alert episode representation without ever interfacing with alerts directly. We also
demonstrate HeAT’s ability to maintain the important attack campaign characteristics defined
by the analyst for multiple networks; where we captured non-coincidental similarities in attacker
strategies between our CPTC and CCDC datasets.</p>
      <p>For the foreseeable future, triaging networks to assess attack campaigns is only going to get
more dificult and time consuming to the point that manual triaging becomes infeasible. As
seen in the related works within the private sector, we believe that automated triaging with
AI/ML will be a necessary component to all SoC operations. We believe that HeAT is a viable
solution that requires minimal expertise and analyst’ efort, and can inspire more research into
AI-based automated triage. We are in the process of making HeAT available and open sourced.
Aside from the future works proposed within our case studies, we plan for HeAT to become
integrated as a plug-in with Splunk or other SIEMs. We also plan to extend HeAT to account
for a variety of intrusion alerts and event logs. Particularly, we consider integrating with Zeek
logs and phishing email detection with HeAT to broaden the insights and address more complex
attack types.
lockheed
martin
security.
[13] S. Moskal and S. J. Yang. Framework to describe intentions of a cyber attack action. arXiv
preprint arXiv:2002.07838, 2020.
[14] S. Moskal, S. J. Yang, and M. E. Kuhl. Extracting and evaluating similar and unique
cyber attack strategies from intrusion alerts. In Proceedings of 2018 IEEE International
Conference on Intelligence and Security Informatics (ISI), pages 49–54. IEEE, 2018.</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          [1]
          <string-name>
            <surname>Alchemer</surname>
          </string-name>
          .
          <article-title>Number of choices in survey questions: How much is too much? https://www</article-title>
          .alchemer.com/resources/blog/survey
          <article-title>-questionshow-much-is-too-</article-title>
          <string-name>
            <surname>much</surname>
            <given-names>/</given-names>
          </string-name>
          ,
          <year>2011</year>
          . [Online; accessed 21-June-2021].
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          [2]
          <string-name>
            <given-names>F. M.</given-names>
            <surname>Alserhani</surname>
          </string-name>
          .
          <article-title>Alert correlation and aggregation techniques for reduction of security alerts and detection of multistage attack</article-title>
          .
          <source>International Journal of Advanced Studies in Computers, Science and Engineering</source>
          ,
          <volume>5</volume>
          (
          <issue>2</issue>
          ):
          <fpage>1</fpage>
          ,
          <year>2016</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          [3]
          <string-name>
            <given-names>Y.</given-names>
            <surname>Chen</surname>
          </string-name>
          ,
          <string-name>
            <given-names>Z.</given-names>
            <surname>Liu</surname>
          </string-name>
          ,
          <string-name>
            <given-names>Y.</given-names>
            <surname>Liu</surname>
          </string-name>
          , and
          <string-name>
            <given-names>C.</given-names>
            <surname>Dong</surname>
          </string-name>
          .
          <article-title>Distributed attack modeling approach based on process mining and graph segmentation</article-title>
          .
          <source>Entropy</source>
          ,
          <volume>22</volume>
          (
          <issue>9</issue>
          ):
          <fpage>1026</fpage>
          ,
          <year>2020</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          [4]
          <string-name>
            <surname>Darktrace</surname>
          </string-name>
          .
          <article-title>Darktrace cyber ai analyst: Autonomous investigations</article-title>
          . https://www.darktrace.com/en/resources/wp-cyber
          <article-title>-ai-analyst</article-title>
          .pdf,
          <year>2021</year>
          . [Online; accessed 21-June-2021].
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          [5]
          <string-name>
            <given-names>S.</given-names>
            <surname>C. De Alvarenga</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Barbon</surname>
          </string-name>
          <string-name>
            <surname>Jr</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R. S.</given-names>
            <surname>Miani</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Cukier</surname>
          </string-name>
          , and
          <string-name>
            <given-names>B. B.</given-names>
            <surname>Zarpelão</surname>
          </string-name>
          .
          <article-title>Process mining and hierarchical clustering to help intrusion alert visualization</article-title>
          .
          <source>Computers &amp; Security</source>
          ,
          <volume>73</volume>
          :
          <fpage>474</fpage>
          -
          <lpage>491</lpage>
          ,
          <year>2018</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          [6]
          <string-name>
            <surname>Fast</surname>
          </string-name>
          .
          <source>AI. Fastai python library v1.0</source>
          .57. http://docs.fast.
          <source>ai/</source>
          ,
          <year>2020</year>
          . [Online; accessed 28-April-2020].
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          [7]
          <string-name>
            <given-names>O. B.</given-names>
            <surname>Fredj</surname>
          </string-name>
          .
          <article-title>A realistic graph-based alert correlation system</article-title>
          .
          <source>Security and Communication Networks</source>
          ,
          <volume>8</volume>
          (
          <issue>15</issue>
          ):
          <fpage>2477</fpage>
          -
          <lpage>2493</lpage>
          ,
          <year>2015</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          [8]
          <string-name>
            <given-names>I.</given-names>
            <surname>Ghafir</surname>
          </string-name>
          ,
          <string-name>
            <given-names>K. G.</given-names>
            <surname>Kyriakopoulos</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Lambotharan</surname>
          </string-name>
          ,
          <string-name>
            <given-names>F. J.</given-names>
            <surname>Aparicio-Navarro</surname>
          </string-name>
          ,
          <string-name>
            <given-names>B.</given-names>
            <surname>AsSadhan</surname>
          </string-name>
          , H. BinSalleeh, and
          <string-name>
            <given-names>D. M.</given-names>
            <surname>Diab</surname>
          </string-name>
          .
          <article-title>Hidden markov models and alert correlations for the prediction of advanced persistent threats</article-title>
          .
          <source>IEEE Access</source>
          ,
          <volume>7</volume>
          :
          <fpage>99508</fpage>
          -
          <lpage>99520</lpage>
          ,
          <year>2019</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          [9]
          <string-name>
            <given-names>F.</given-names>
            <surname>Hassanabad</surname>
          </string-name>
          .
          <article-title>suricata-sample-data</article-title>
          . https://github.com/FrankHassanabad/suricatasample-data/blob/master/README.md,
          <year>2019</year>
          . [Online; accessed 5-May-2020].
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          [10]
          <string-name>
            <surname>IBM</surname>
          </string-name>
          .
          <article-title>Ibm qradar advison with watson</article-title>
          . https://www.ibm.com/products/ cognitive-security-analytics,
          <year>2021</year>
          . [Online; accessed 21-June-2021].
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          [11]
          <string-name>
            <given-names>M.</given-names>
            <surname>Landauer</surname>
          </string-name>
          ,
          <string-name>
            <given-names>F.</given-names>
            <surname>Skopik</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Wurzenberger</surname>
          </string-name>
          ,
          <string-name>
            <given-names>W.</given-names>
            <surname>Hotwagner</surname>
          </string-name>
          ,
          <article-title>and</article-title>
          <string-name>
            <given-names>A.</given-names>
            <surname>Rauber</surname>
          </string-name>
          .
          <article-title>A framework for cyber threat intelligence extraction from raw log data</article-title>
          .
          <source>In Proceedings of 2019 IEEE International Conference on Big Data (Big Data)</source>
          , pages
          <fpage>3200</fpage>
          -
          <lpage>3209</lpage>
          . IEEE,
          <year>2019</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          [12]
          <string-name>
            <given-names>L.</given-names>
            <surname>Martin</surname>
          </string-name>
          . Cyber kill chain | http://cyber.lockheedmartin.com/solutions/ cyber-kill-chain,
          <year>2016</year>
          . [Online; accessed 11-April-2016].
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          [15]
          <string-name>
            <given-names>A.</given-names>
            <surname>Nadeem</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Verwer</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Moskal</surname>
          </string-name>
          , and
          <string-name>
            <given-names>S. J.</given-names>
            <surname>Yang</surname>
          </string-name>
          .
          <article-title>Alert-driven attack graph generation using s-pdfa</article-title>
          . Accepted to Appear
          <source>in 2021 ACM KDD Workshop on Artificial Intelligenceenabled Cybersecurity Analytics (AI4Cyber)</source>
          ,
          <year>2021</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          [16]
          <string-name>
            <given-names>J.</given-names>
            <surname>Navarro</surname>
          </string-name>
          ,
          <string-name>
            <given-names>V.</given-names>
            <surname>Legrand</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Lagraa</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>François</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Lahmadi</surname>
          </string-name>
          ,
          <string-name>
            <surname>G. De Santis</surname>
            ,
            <given-names>O.</given-names>
          </string-name>
          <string-name>
            <surname>Festor</surname>
            ,
            <given-names>N.</given-names>
          </string-name>
          <string-name>
            <surname>Lammari</surname>
            ,
            <given-names>F.</given-names>
          </string-name>
          <string-name>
            <surname>Hamdi</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          <string-name>
            <surname>Deruyver</surname>
          </string-name>
          , et al.
          <article-title>Huma: A multi-layer framework for threat analysis in a heterogeneous log environment</article-title>
          .
          <source>In International Symposium on Foundations and Practice of Security</source>
          , pages
          <fpage>144</fpage>
          -
          <lpage>159</lpage>
          . Springer,
          <year>2017</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref15">
        <mixed-citation>
          [17]
          <string-name>
            <given-names>J.</given-names>
            <surname>Navarro</surname>
          </string-name>
          ,
          <string-name>
            <given-names>V.</given-names>
            <surname>Legrand</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Deruyver</surname>
          </string-name>
          , and
          <string-name>
            <given-names>P.</given-names>
            <surname>Parrend</surname>
          </string-name>
          .
          <article-title>Omma: open architecture for operator-guided monitoring of multi-step attacks</article-title>
          .
          <source>EURASIP Journal on Information Security</source>
          ,
          <year>2018</year>
          (1):
          <fpage>1</fpage>
          -
          <lpage>25</lpage>
          ,
          <year>2018</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref16">
        <mixed-citation>
          [18]
          <string-name>
            <given-names>C.</given-names>
            <surname>Networks.</surname>
          </string-name>
          Ai-analyst. Centrcentripetalnetworks.com/hubfs/Data%20Sheets /CI_AI_Analyst_Brief.pdf,
          <year>2018</year>
          . [Online; accessed 21-June-2021].
        </mixed-citation>
      </ref>
      <ref id="ref17">
        <mixed-citation>
          [19]
          <string-name>
            <surname>R. I.</surname>
          </string-name>
          of Technology.
          <source>Index of cptc2018</source>
          . http://mirror.rit.edu/cptc/2018/,
          <year>2021</year>
          . [Online; accessed 21-June-2021].
        </mixed-citation>
      </ref>
      <ref id="ref18">
        <mixed-citation>
          [20]
          <string-name>
            <given-names>X.</given-names>
            <surname>Qin</surname>
          </string-name>
          and
          <string-name>
            <given-names>W.</given-names>
            <surname>Lee</surname>
          </string-name>
          .
          <article-title>Discovering novel attack strategies from infosec alerts</article-title>
          .
          <source>In Proceedings of 2004 European Symposium on Research in Computer Security</source>
          , pages
          <fpage>439</fpage>
          -
          <lpage>456</lpage>
          . Springer,
          <year>2004</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref19">
        <mixed-citation>
          [21]
          <article-title>Rochester Institute of Technology. Collegiate penetration testing competition</article-title>
          . hhttp://nationalcptc.org,
          <year>2018</year>
          . [Online; accessed 19-July-2018].
        </mixed-citation>
      </ref>
      <ref id="ref20">
        <mixed-citation>
          [22]
          <string-name>
            <surname>C.-H. Wang</surname>
            and
            <given-names>Y.-C.</given-names>
          </string-name>
          <string-name>
            <surname>Chiou</surname>
          </string-name>
          .
          <article-title>Alert correlation system with automatic extraction of attack strategies by using dynamic feature weights</article-title>
          .
          <source>International Journal of Computer and Communication Engineering</source>
          ,
          <volume>5</volume>
          (
          <issue>1</issue>
          ):
          <fpage>1</fpage>
          ,
          <year>2016</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref21">
        <mixed-citation>
          [23]
          <string-name>
            <given-names>L.</given-names>
            <surname>Wang</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Liu</surname>
          </string-name>
          , and
          <string-name>
            <given-names>S.</given-names>
            <surname>Jajodia</surname>
          </string-name>
          .
          <article-title>Using attack graphs for correlating, hypothesizing, and predicting intrusion alerts</article-title>
          .
          <source>Computer communications</source>
          ,
          <volume>29</volume>
          (
          <issue>15</issue>
          ):
          <fpage>2917</fpage>
          -
          <lpage>2933</lpage>
          ,
          <year>2006</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref22">
        <mixed-citation>
          [24]
          <string-name>
            <surname>WRCCDC</surname>
          </string-name>
          .
          <article-title>Wrccdc public archive</article-title>
          . https://archive.wrccdc.org/,
          <year>2021</year>
          . [Online; accessed 21-June-2021].
        </mixed-citation>
      </ref>
      <ref id="ref23">
        <mixed-citation>
          [25]
          <string-name>
            <given-names>B.</given-names>
            <surname>Zhu</surname>
          </string-name>
          and
          <string-name>
            <given-names>A. A.</given-names>
            <surname>Ghorbani</surname>
          </string-name>
          .
          <article-title>Alert correlation for extracting attack strategies</article-title>
          .
          <source>IJ Network Security</source>
          ,
          <volume>3</volume>
          (
          <issue>3</issue>
          ):
          <fpage>244</fpage>
          -
          <lpage>258</lpage>
          ,
          <year>2006</year>
          .
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>