<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>Modelling Hybrid Cyber Kill Chain</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Wen Zeng</string-name>
          <email>e@s</email>
          <email>e@s0</email>
          <email>wen.zeng.wz@gmail.com</email>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Vasileios Germanos</string-name>
          <email>l@s</email>
          <email>l@s0</email>
          <email>vasileios.germanos@gmail.com</email>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>Cyber Technology Institute, School of Computer Science and Informatics De Montfort University</institution>
          ,
          <addr-line>Leicester LE1 9BH</addr-line>
          ,
          <country country="UK">U.K</country>
        </aff>
      </contrib-group>
      <fpage>143</fpage>
      <lpage>160</lpage>
      <abstract>
        <p>The Cyber Kill Chain (CKC) model is an industry-accepted methodology for understanding how an attacker will conduct the activities necessary to cause harm to the organizations. CKC has seen some adoption in the information security community. However, acceptance is not universal, with critics pointing to what they believe are fundamental flaws in the model. Therefore, an effective understanding of the CKC will greatly assist the information security professionals in establishing strong controls and countermeasures that will serve to protect organizations' assets. In this paper, we will develop a new framework - Hybrid Cyber Kill Chain which is based on CKC, and then a formal model will be built to analyse this framework. In this framework, we will investigate the strength level of threat actors and defenders. The framework can be captured by coloured Petri nets. This study can help security professionals find out the fundamental flaws of the basic CKC. Our work also can help business organizations take actions to deal with incidents to eliminate or minimize the impact and establish strong controls and countermeasures.</p>
      </abstract>
      <kwd-group>
        <kwd>Petri nets</kwd>
        <kwd>Cyber security</kwd>
        <kwd>controls</kwd>
        <kwd>Cyber Kill Chain</kwd>
        <kwd>Cyber attack</kwd>
      </kwd-group>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>Introduction</title>
      <p>
        Cyber Kill Chain (CKC) is developed by Lockheed Martin, which is an
american global aerospace, defense, security and advanced technologies company
(https://www.lockheedmartin.com) [
        <xref ref-type="bibr" rid="ref12">12</xref>
        ]. CKC is an intelligence-driven,
threatfocused framework (see Figure 1) for the defence of computer networks. It takes
the view point of an adversary, which shows the stages of what must be achieved
in order for an attacker to be successful. This could give network defenders
the advantage in fighting cyber attackers by establishing strong controls and
countermeasures. The cyber kill chain model has seen some adoption in the
information security community [
        <xref ref-type="bibr" rid="ref14">14</xref>
        ]. However, acceptance is not universal, with
critics pointing to what they believe are fundamental flaws in the model [
        <xref ref-type="bibr" rid="ref19">19</xref>
        ].
Therefore, it is necessary to build a formal model to analyse the CKC which can
help us find out the fundamental flaws.
      </p>
      <p>
        Recently, there is a significant interest in analysing CKC. In Black Hat 2013,
Patrick Reidy presented a variety of methods to combat insider threats at the FBI
[
        <xref ref-type="bibr" rid="ref21">21</xref>
        ]. These methods are based on CKC framework. It was stated that although
CKC is an excellent framework for the defence of external attacks, it is not so
suitable for insider threats. In general, there is a misunderstanding regarding
the type of threats in an organisation. It is often the case that a threat is not
an external one (e.g., some hacker) but an insider (e.g., people who joined the
organisation) [
        <xref ref-type="bibr" rid="ref21">21</xref>
        ]. Dealing with insider threats is not a task that can be supported
by firewalls and antivirus. This is not a simple cyber security problem and these
defence mechanisms do not apply as no rules are being broken. As a result, they
proposed a multidisciplinary approach that extends the CKC framework, the
Insider Threat CKC.
      </p>
      <p>
        In [
        <xref ref-type="bibr" rid="ref6">6</xref>
        ], the author argued that the scope of today’s cyber threats extends far
beyond that of CKC. Today’s networks are facing various threat vectors, e.g.,
social engineering; insider threat; and intrusion based on remote access. All these
threat vectors are not related to malware or payload. However, CKC is based on
perimeter/malware-focused thinking. Moreover, on an attack time scale, the steps
of CKC (see Section 3) are disproportionate. The steps from Reconnaissance to
Command and Control comprise a small part of an attack, compare to the last
two steps (“Command &amp; Control" and “Action on Objectives") which are the
biggest two. Consequently, most of the attention must be paid between these
two last steps. The reason behind this is that once the attacker penetrates the
perimeter defined by CKC, the prevention solutions, e.g., firewalls and antivirus,
cannot be applied. Thus, one should focus on deploying a breach detection system
that automatically detects and analyses any change in user/computer behaviour
that indicates a breach.
      </p>
      <p>
        In [
        <xref ref-type="bibr" rid="ref15">15</xref>
        ], the author presented three key benefits of applying the CKC model.
Firstly, CKC enhances the understanding of the capabilities an organisation has;
any gaps that are not covered by tools; and threat actors. Secondly, CKC can be
leveraged in post-incintent analysis by break down the attack in a systematic
way. Thirdly, a big advantage of CKC model is that it can provide, efficiently,
an explanation about how a complex situation evolved, using use-case diagrams.
However, there is some criticism about CKC ability to handle an insider breach
[
        <xref ref-type="bibr" rid="ref21">21</xref>
        ]. In this case, CKC framework can be effectively used as an insider, who
would conduct the same steps of CKC having as a final goal to achieve the
“Actions on Objectives" stage. The US Senate Committee on Commerce, Science,
and Transportation published a report [
        <xref ref-type="bibr" rid="ref28">28</xref>
        ] regarding the analysis of a cyber
attack against Target, one of the largest retail companies in the US. In this case,
the attacker managed to gain access to Target’s network using as an insider a
third-party vendor of the company. The analysis of this report was conducting
using the principles of CKC. All in all, CKC is a powerful framework that can
resolve highly complex attack scenarios and be customised/extended, accordingly,
to fit an organisation’s specific requirements.
      </p>
      <p>
        In [
        <xref ref-type="bibr" rid="ref11">11</xref>
        ], Laliberte proposed a version of CKC aiming to model types of attacks
that take advantage of malicious websites to target outdated software on its
visitors systems. This version proposed an additional stage the Lateral Movement,
which sits between Command &amp; Control and Actions on Objectives. This new
stage expresses the move of the attacker to bigger targets using the target network
after some of its systems has been compromised. In addition, the author states
that one cannot provide defence mechanisms against the Weaponisation stage,
so this stage is redundant and is not considered in the proposed kill chain.
      </p>
      <p>
        In [
        <xref ref-type="bibr" rid="ref20">20</xref>
        ], Nachreiner proposed a kill chain where Weaponisation stage is not
considered as well for the same reason as in [
        <xref ref-type="bibr" rid="ref11">11</xref>
        ]. The Installation stage is replaced
by Infection with the intuition that attackers can use alternative methods apart
from installation. To capture the way a malicious code can spread in the target’s
network by leveraging its initial network access, it is proposed to embed the
notions of Lateral Movement and Pivoting within the Actions on Objectives
stage. The term pivoting describes the exploitation of a system to connect other
external or internal systems, in which direct access is not feasible.
      </p>
      <p>
        MITRE, a non-profit research centre, proposed a more structured framework
that can improve the analysis of cyber attacks, the Adversarial Tactics,
Techniques &amp; Common Knowledge for Enterprise (ATT&amp;CK) [
        <xref ref-type="bibr" rid="ref17">17</xref>
        ]. In a nutshell, this
framework consists of stages in a non-sequential style to provide more detailed
information regarding the actions an attacker performs once inside the target’s
network. Among other stages, ATT&amp;CK introduces the Persistence stage, which
denotes the attacker’s persistence to accomplish an action.
      </p>
      <p>The aim of this paper is to develop a framework to analyse how attackers
conduct the activities to organizations based on CKC, and then the proposed
framework will be formally modelled and analysed. The strength level will be
introduced to attackers and defenders in the model, and then we will demonstrate
how Petri net techniques can be used to model the proposed framework.</p>
      <p>This paper is organized as follows. Section 2 is related work. Section 3 provides
the basic definitions used throughout this paper. In Section 4, we will introduce
a new framework for CKC - Hybrid Cyber Kill Chain (HCKC), and a formal
model will be developed for HCKC. The basic definitions relating to Petri nets
are given in Section 5. Section 6 outline how Petri nets could be used to support
analysing the proposed framework. Section 7 concludes the paper.
2</p>
    </sec>
    <sec id="sec-2">
      <title>Related Work</title>
      <p>Large organizations use a large number of business processes, which in many
cases are unique, to achieve their goals and objectives. Moreover, their concurrent
structure and complexity is a challenge for system architects and security analysts
to develop efficient and secure systems. To alleviate their workload various
modelbased security analysis techniques have been proposed.</p>
      <p>
        Our study is not the first instance of model-based security analysis of hackers’
behaviours. In [
        <xref ref-type="bibr" rid="ref23">23</xref>
        ], attack trees have been used to describe how sets of events can
constitute a security compromise. Attack trees are useful for thinking multiple
ways that a hacker can reach an attack goal. In [
        <xref ref-type="bibr" rid="ref24 ref25">24, 25</xref>
        ], the authors introduced
attack graphs which were extended to attack trees by introducing state to analysis.
Although, attack trees enable state-based analysis, they do not take into account
the different attack goals and preferences of individual attackers. Attack trees
and attack graphs are popular modelling approaches because they are good at
describing an attack in an intuitive visual way, show all attack paths within
a broad picture, and can lead to useful mathematical analysis if in the nodes
are assigned values. However, attack trees and attack graphs are proceeding in
sequential steps. They focus on a single goal and a single attacker.
      </p>
      <p>
        In [
        <xref ref-type="bibr" rid="ref16">16</xref>
        ], McDermott proposed that Petri nets are more appropriate for cyber
attack modelling than attack trees, as Petri nets do not have the limitations of
attack trees and attack graphs, and can capture complex attacks with multiple
steps. In addition, Petri nets are better at capturing concurrent actions in the
progression of an attack. For instance, multiple instances of attackers and attack
goals can be captured and analysed. In [
        <xref ref-type="bibr" rid="ref29">29</xref>
        ], the authors proposed that coloured
Petri nets are suitable for hierarchical attack modelling, as they more expressive.
      </p>
      <p>
        There are some existing work aimed at analyzing security properties using
Petri nets: In [
        <xref ref-type="bibr" rid="ref2 ref3">3, 2</xref>
        ], the authors proposed a Petri net modelling technique can
be used to analyse opacity which is a security property in distributed systems.
In [
        <xref ref-type="bibr" rid="ref27">27</xref>
        ], the author proposed an alternative method of information system risk
analysis that supports ongoing information system risk management based upon
information system security modelling. Petri nets are used to simulate and
analyse the complex information security system behaviour. In [
        <xref ref-type="bibr" rid="ref30">30</xref>
        ], the authors
proposed a formal threat-driven approach which explores explicit behaviours
of security threats as the mediator between security goals and application of
security feature. The security threats are modelled by Petri nets which can verify
correctness of security threats against intended functions and verify absence
of security threats from integrated functions and threat mitigation. In [
        <xref ref-type="bibr" rid="ref4">4</xref>
        ], the
authors modelled cyber-physical attacks on smart grid using Petri nets, in this
study, they proposed a hierarchical method to construct large Petri nets from a
number of small Petri nets that can be created separately by different domain
experts. In [
        <xref ref-type="bibr" rid="ref33">33</xref>
        ], the authors proposed a Petri net model to analyse the threats
and user behaviour in the business organizations. This study focused on the user
behaviour, threats and security policies in a dynamic environment. In [
        <xref ref-type="bibr" rid="ref9">9</xref>
        ], the
authors proposed an approach to model malware behaviour using coloured Petri
nets. In [
        <xref ref-type="bibr" rid="ref10 ref26">10, 26</xref>
        ], the authors demonstrated how coloured Petri nets can be used to
model protocol elements and improve protocol specifications, and how state space
exploration can be verify protocol properties. In [
        <xref ref-type="bibr" rid="ref32 ref34">34, 32</xref>
        ], the authors proposed
that Petri nets related techniques can be used to analyse the information flow
security in cloud computing systems. In [
        <xref ref-type="bibr" rid="ref31">31</xref>
        ], the author proposed a Petri net
based methodology to analyse the non-productive time in the organizations by
implementing information security technologies.
3
      </p>
    </sec>
    <sec id="sec-3">
      <title>Preliminary Material</title>
      <p>In this part, the definition of CKC is presented making it easier to follow the
technical content of this paper.</p>
      <p>
        CKC is a framework that helps to understand how a threat actor, such as
an attacker, will orchestrate the necessary actions to cause harm to an
organisation. There are seven stages in CKC: Reconnaissance, weaponization, delivery,
exploitation, installation, command &amp; control, and action on objectives [
        <xref ref-type="bibr" rid="ref12 ref5">12, 5</xref>
        ].
      </p>
      <p>Stage 1 - Reconnaissance: In this stage, the attackers would gather as much
information as possible about target organization. For example: names of the
employers, their positions, email addresses and IP addresses. They also can
purchase information security resources in State and Military level.</p>
      <p>Stage 2 - Weaponisation: In this stage, the attackers develop exploitation and
malware. The attackers might create websites that contain malware, and develop
malicious software for a specific platform or purpose (e.g., a vulnerable Adobe
Acrobat (PDF) or a Microsoft Office (DOC) file), which designed according to
the vulnerabilities discovered during reconnaissance.</p>
      <p>Stage 3 - Delivery : In this third stage, the attackers transmits the malicious
code to the target information system. The attacker might use a spear-phishing
attack targeting an internal employee of the organisation, social engineering or
some vulnerability of the system to deliver the malware.</p>
      <p>Stage 4 - Exploitation: The attacker in this phase, utilizing the discovered
vulnerabilities, is executing the malicious code on the target network using
remote or local mechanisms. The aim is to gain superuser access to the target’s
information system.</p>
      <p>Stage 5 - Installation: As soon as the exploitation is successful, the malicious
code will install itself onto the target’s information system. Then, the malicious
code will start downloading additional software when there is available network
access. The attacker can maintain continuity of access to the target system by a
remote access trojan or creating a backdoor in order to penetrate further into
the target network. This practice keeps the delivery payload small making it
undetectable.</p>
      <p>Stage 6 - Command &amp; Control : This stage is also know as C2. Here, the
attacker has set up the management and communication code onto the target
network. Now, the attacker can fully manage the malicious code and move further
into the network, exfiltrate data and conduct destruction or denial of service
operations.</p>
      <p>Stage 7 - Action on Objectives: The actions and objectives of the attacks are
dependent on their specific mission (e.g., exfiltrate data and conduct destruction
or denial of service operations).</p>
      <p>An assimilation of CKC will greatly assist chief information security officers
in establishing strong controls and countermeasures that will serve to protect
their organization’s assets.
4</p>
    </sec>
    <sec id="sec-4">
      <title>Hybrid Cyber Kill Chain</title>
      <p>In this section, we will introduce a new framework – Hybrid Cyber Kill Chain
(HCKC).</p>
      <p>
        In general, the purpose of CKC is to enhance the resilience of an organisation
against cyber attacks. Although CKC has been widely adopted by organisations
since it has been published, it has been criticized for its weakness in handling
insider threats, because it is too focused on the perimeter and malware prevention
[
        <xref ref-type="bibr" rid="ref21">21</xref>
        ]. This statement is supported also by case studies, where CKC has been
applied. For example, [
        <xref ref-type="bibr" rid="ref28">28</xref>
        ] shows that the stages of CKC do not capture all
possible attack vectors and are not expressive enough to model internal actions
within the target network. Consequently, based on these facts, improvements and
amendments to CKC have been proposed. Some of these proposed extensions try
to address the insider’s perspective.
      </p>
      <p>On top of CKC, the proposed kill chain framework aims to provide a holistic
approach by taking into account the following factors:
- The persistence and the lateral movement of the attacker;
- The defence actions of the defender;
- The strength level both of the attacker and defender.</p>
      <p>The third factor is embedded in each stage of HCKC after the weaponisation.
That is because the two first stages are not under the control of the defender.
According to this rule, the attacker’s strength level must be greater than the one
of the defender’s in order to proceed to the next stage in the kill chain.</p>
      <p>HCKC consists of two parts. Initially, we have the CKC augmented with extra
two stages – Persistence and Lateral Movement. Then, we map to each attack
stage an extra stage that corresponds to the defensive actions of the defender, as
depicted in Figure 2. Moreover, a strength level is assigned to each stage.</p>
      <p>In practice, HCKC is not only useful for post-incident analysis of attacks.
A formal model of the HCKC can provide security metrics that can be utilized
by system architects to make trade-off decisions involving system security. For
instance, there is a growing need for efficient and timely incident response, as
the number of targeted and complex cyber attacks on business organizations is
increasing. This need becomes more urgent as the information collecting from
intrusion detection systems often must be processed manually to enable decisions
on how to respond for thwarting attacks. In many cases, the period of time
between detection and response can be months long, and can allow adversaries to
attain their goals. To that end, the exploitation of the assigned strength levels and
defenders can contribute to the development/utilization of automated incident
response techniques, security metrics, and verification properties.
4.1</p>
      <sec id="sec-4-1">
        <title>Strength Levels in HCKC</title>
        <p>Throughout the paper, we will assume that the basis of a HCKC is a set S of
discrete stages in HCKC. Moreover, E will denote hackers, includes internal and
external hackers, which are entities in the model. We now assign the strength
level to any entity, which will in practice be related to the degree of strength of
its entity.</p>
        <p>A lattice for strength concerns, L = (L,  ) consists of a set L and a partial
order relation  such that, for all l, l0 2 L, there exists a least upper bound
l t l0 2 L, and a greatest lower bound l u l0 2 L. The lattice is complete if each
subset L of L has both a least upper bound ` L and a greatest lower bound
Q L. We will assume that the lattice L is complete.</p>
        <p>A strength level map: ` : E ! L. This represents the strength level of each
hacker.</p>
        <p>Firstly, we assign strength levels to stages, which represent the defenders’
strength level of each stage:
Moreover, a new mapping loc is used to return the stage of each entity:
- ` : S ! L
- loc : E ! S
Then we add a rule that an entity can only be deployed in a stage with a strength
level that is smaller than that of the entity. That is, for each entity e: an entity e
is located at stage s, then we must have
`(loc(e)) &lt; `(e)
(1)
4.2</p>
      </sec>
      <sec id="sec-4-2">
        <title>A Model for HCKC</title>
        <p>We now introduce a formal model for capturing the behaviour of HCKC. The
proposed model uses tuples to represent entities in HCKC. Each such tuple
comprises information about the nature of the entities (hackers’ behaviour). Since
there can be duplicates of hackers within the same stage of HCKC, the state
of the system is a multiset of entities, allowing for an arbitrary multiplicity
of any hackers. The transformations of the system are then defined through
the simultaneous execution of individual actions, each action being executed
instantaneously and possibly many times.</p>
        <p>The system is based on a fixed set of stages. To aid the understanding of
the system model, it is introduced in three steps. First, we specify the overall
structure in Definition 1. Then, in Definition 2, we introduce rules that explain
the transformation between the system states. Finally, we specify the exact format
of hackers’ actions supported by the model.</p>
        <p>Definition 1 (HCKC structure). A model for Hybrid Cyber Kill Chain is a
tuple:</p>
        <p>HCKC = (S, E, L, `, A, stinit ) ,
(2)
where: S is a finite nonempty set of stages; E is a finite nonempty set of hackers;
L is a complete strength lattice; ` : S ! L is a mapping assigning strength levels
to the stages; A is a finite set of actions, each action being a pair = ( in, out)
consisting of two finite multisets over the set of tuples</p>
        <p>M = (E ⇥ L ⇥ S);
and stinit is an initial state defined as a finite multiset over M. In general, a
state of HCKC is a finite multiset over M, and x 2 M is present in a state st
if st(x) &gt; 0.</p>
        <p>A tuple (e, l, s) 2 M , denoted by (e, l)@s, represents a hacker e with the
strength level l at stage s. An entity can have several different copies, and each
of these copies can have a different strength level and may appear at different
stage. As already mentioned, we allow multiple copies of a single entity to be
present in a stage. Hence a state is a multiset st over M rather than a subset
of M. For example, st8(e6, 1, s4) = 4 means that in the state st8 there are four
copies of hacker e6 with strength level 1 at stage s4.</p>
        <p>Now we define how the system can proceed from one state to another state
by executing a multiset of actions.</p>
        <p>It is assumed that the executed (instances of) actions cannot share input
entities. For example, if there is one copy of an entity present, then at most
one action which has this entity in its input can be executed. This results in
conflicts between actions which could potentially be executed, and contributes to
nondeterminism in system execution. The formal semantics, and then property
verification, take into account all possible ways in which such conflicts could be
resolved.</p>
        <p>Below ( ) and (+) are respectively the multiset subtraction and addition
operations.</p>
        <p>Definition 2 (HCKC semantics). A multiset = { 1, . . . , n} of actions
over A, where i = ( iin , iout ) for i = 1, . . . , n, is enabled at state st if
in =
i1n + . . . + in</p>
        <p>n  st .
Then
can then be executed leading to a state st0 given by:
in +
st0.
st0 = st
out = st
in +
out + . . . +
1
out .
n
We denote this by st !</p>
        <p>With such a definition we can state precisely what are the states which can
be reached from the initial one.</p>
      </sec>
      <sec id="sec-4-3">
        <title>Definition 3 (HCKC reachable states). The reachable states of HCKC in</title>
        <p>Definition 1 is the minimal set of states RS containing stinit such that if st 2 RS
and st !</p>
        <p>st0, for some multiset of actions , then st0 2 RS.</p>
        <p>Hackers’ Actions Hackers can move between different stages in the chain, so
the actions concern the movement of the hackers in different stages. Each action
= ( in , out ) 2 A is such that:
(3)
where s, s0 2 S, e 2 E, and l 2 L.</p>
        <p>Note also that in practice the actions in A can be specified in a more convenient
way; for example, by using guards and parameters. This is illustrated in the Petri
net representation discussed later in this paper, where net transitions use guards
and arcs use parameters (variables).</p>
        <p>Remark 1. The system model introduced above has been kept deliberately simple.
This should allow one to define on top of it a variety of user-friendly, and thus
more practical, notations for system specification and property verification. We
will demonstrate in the rest of this paper how this can be achieved.</p>
        <p>Despite its relative simplicity, the model is very expressive and yet tractable.
Basically, it is equivalent to the model of Petri nets introduced in Section 5,
where state reachability is decidable.</p>
        <p>One could then ask what would happen if we increased the modelling power
of the basic system model. A possible extension could allow, for example, to
check for the absence of certain kinds of hackers. This would lead, in terms of
Petri nets, to the introduction of inhibitor arcs and a loss of the decidability of
state reachability as Petri nets extended by inhibitor arcs are Turing powerful.
The same would be the case if the execution model assumed that at each step a
maximal multiset of action was executed.</p>
        <p>Therefore, as one of our aims is to keep the system model tractable, we believe
that the formalisation presented above strikes a right balance between being
useful for practical applications and amenable to automated verification.
Well-formedness In addition to verifying the property of the model, there are
other desirable functional properties which one would normally need to verify
using, e.g., model checking tools. The following are examples of such properties
formulated for HCKC in (2):
- HCKC is live, if for every reachable state st0 and every action , there is a
state st00 reachable from st0 at which can be executed.
- HCKC is bounded, if there is n
is less than n.</p>
        <p>1 such that the size of each reachable state
- HCKC is well-formed, if there is a state st such that HCKC is both live and
bounded after replacing stinit by st.
5</p>
      </sec>
    </sec>
    <sec id="sec-5">
      <title>Petri Nets</title>
      <p>
        Petri nets are a graphical modelling tool for a formal description of systems whose
dynamics are characterized by concurrency, synchronization, mutual exclusion
and conflict [
        <xref ref-type="bibr" rid="ref13 ref18">13, 18</xref>
        ].
5.1
      </p>
      <sec id="sec-5-1">
        <title>Place/Transition Nets</title>
        <p>A Place/Transition net (PTN) N consists of two disjoint finite sets of nodes, P l
and T r, respectively called places and transitions, a mapping W : (P l ⇥ T r) [
(T r ⇥ P l) ! N specifying the weights of arcs that connect the nodes, and the
initial marking (state) M0 : P l ! N . In general, any finite multiset of places is a
marking (or state) of N .</p>
        <p>Intuitively, places carry (black) tokens which represent the current distribution
of resources in a system modelled by the net. In other words, the current state of
the modelled system is given by the number of tokens in each place.</p>
        <p>Transitions are the active components of the net. An input arc of a transition
tr starts at a place pl and ends at tr provided that n = W (pl, tr) &gt; 0. In such a
case, n is the arc’s weight signifying that an execution of tr requires n tokens in
pl which are consumed as a result. Similarly, an output arc from tr to pl exists
provided that m = W (tr, pl) &gt; 0, and an execution of tr inserts m tokens into pl.</p>
        <p>A transition tr is allowed to be executed (or fired ) at a marking M if M (pl)
W (pl, tr), for all places pl. Its firing produces a new marking M 0 such that
M 0(pl) = M (pl) W (pl, tr) + W (tr, pl), for all places pl. In general, one can
fire a finite multiset of transitions U = {tr1, . . . , trk} provided that M (pl)
W (pl, tr1) + · · · + W (pl, trk), for all places pl (that is, input tokens cannot be
shared), and its firing results in a marking M 0 such that M 0(pl) = M (pl)
W (pl, tr1) · · · W (pl, trk) + W (tr1, pl) + · · · + W (trk, pl), for all places pl. One
can then consider finite and infinite execution sequences starting from the initial
marking, and introduce the notion of a reachable marking.</p>
        <p>
          Petri nets, in particular PTNs, have been widely used for structural modelling
of workflows and have been applied in a wide range of qualitative and quantitative
analyses (for example, [
          <xref ref-type="bibr" rid="ref1 ref13 ref22">13, 22, 1</xref>
          ]).
5.2
        </p>
      </sec>
      <sec id="sec-5-2">
        <title>Coloured Petri Nets</title>
        <p>PTNs are a low-level model, and in practical applications, it is convenient to use
more compact (but behaviourally equivalent) high-level Petri net models. An
example of such a compact model are coloured Petri nets (CPNs), where the
tokens are tuples of values, the arcs are used as selectors allowing one to specify
the format of input and output tokens, and transitions have associated guards
which allow one to easily express, e.g., various security policies.</p>
        <p>Let Tok be a finite set of elements (or colours) and VAR be a disjoint finite
set of variable names. In a CPN:
- Each place has a type, which is a subset of Tok indicating the colour of tokens
this place can contain. A marking is obtained by placing in each place a
multiset of tokens belonging to the type of the place.
- Each arc is labelled with a multiset of variables from VAR.
- Each transition has a guard, which is a Boolean expression over Tok [ VAR.</p>
        <p>For a transition t, VAR(t) denotes the set of variables appearing in its guard
and labelling its input and output arcs.</p>
        <p>The enabling and firing rules of coloured Petri Nets are as follows: when
tokens flow along the incoming arcs of a transition t, they become bound to
variables labelling those arcs, forming a binding mapping : VAR(t) ! Tok. If
this mapping can be extended to a total mapping 0 in such a way that the guard
of t evaluates to true and the values of the variables on the outgoing arcs are
consistent with the types of the places these arcs point to, then t is enabled and 0
is an enabling binding of t. An enabled transition can fire, consuming the tokens
from its pre-set and producing tokens in places in its post-set, in accordance with
the values of the variables on the appropriate arcs given by 0. One can then
define an enabling condition and firing rule for a multiset of transitions with
enabling bindings similarly as it was done for PTNs, and introduce notions like
marking reachability by generalizing those defined for PTNs.
6</p>
      </sec>
    </sec>
    <sec id="sec-6">
      <title>Hybrid Cyber Kill Chain in Coloured Petri Nets</title>
      <p>We now outline how CPNs could be used to represent (and then used to verify)
a given HCKC. We will now use a scenario (Target Data Breach) to illustrate
the definition of HCKC model, and the way it can be represented using coloured
Petri nets.</p>
      <p>
        In 2013, Target, one of the largest retail companies in the US, was a victim
of cyber attack [
        <xref ref-type="bibr" rid="ref28">28</xref>
        ]. The attackers surreptitiously managed to gain access to
Target’s network, stole the financial and personal information of 110 million
customers, and then removed this sensitive data from Target’s network to a server
somewhere in Eastern Europe. Below they are outlined the key points at which
Target failed to detect and prevent the attack:
- Target enabled a third-party vendor to have access to their network. The
vendor did not follow appropriate information security practices. As a result,
the weak security of the vendor allowed the attackers to gain access in Target’s
network.
- While the attackers were installing malicious software on Target’s system,
the company failed to respond to a series of automated warnings from their
intrusion detection software.
- Attackers managed to move from less sensitive areas of Target’s network to
areas storing consumer data by leveraging vendor’s credentials.
- Target failed to respond to a series of warnings from their intrusion detection
software, while the attackers removed Target’s data via the network.
      </p>
      <p>For this scenario, we have in total nine stages and one entity (hacker). Stages,
si : i 2 { 0, ..., 8}, are sorted according to the position of each stage in HCKC (see
Figure 2), thus s0 stands for Reconnaissance and s8 for Action on Objectives.
Moreover, based on HCKC, a defence mechanism is mapped to each stage, the
purpose is to prevent a hacker to move to next stages in the kill chain. In both, the
hacker and defence mechanisms, are assigned a strength level. A higher strength
level for a hacker means that is more capable to pass each stage in the kill chain,
and for a defence mechanism to block an attack. A hacker can be deployed on
different stages.</p>
      <p>The strength levels of entities and defence level of different stages are listed in
Table 1. In this scenario, the hacker initially attacked the vendor’s weak network
system and used it to gain access in Target’s network. For this reason, two kill
chains are combined together to describe how Target’s cyber attack evolved. Stage
s1, ... , s8 belong to the kill chain for the vendor’s attack, while for the Target’s
attack we have the stages s04, ... , s0 . That is because the hacker’s aim was to
8
reach the Lateral Movement in vendor’s kill chain (i.e., s7). Then, the attacker
gained access to Target’s network. The kill chain of the attack, when the attacker
managed to enter the Target’s network starts from stage s04 (i.e., Exploit), as the
attacker was already in the Target’s network (the previous stages are applied
once an attacker is outside the network). The hacker can pass to a next stage
only if he/she has higher strength level than a defence mechanism. Here, as the
hacker executed a successful cyber attack, the value of his/her strength level is
higher (l = 2) than the defence mechanisms of all stages.</p>
      <p>Table 2 shows all the valid mappings of entities to stages. We can observe,
e.g., that e0 can be deployed at stage s1, . . . , s8, s04, . . . , s08.</p>
      <p>The structure and dynamics of the system are represented by the coloured
Petri net in Figure 3. In this diagram, tokens represent entities; places represent
different stages; and transitions capture the activities. s, l, etc., are parameters
(variables).</p>
      <p>We assume that in the initial state of the system there is one hacker e0
residing at stage s0. The hacker is represented by a token in place labelled as
s0 being (e0, 2)@s0. This net shows how the hacker can move between different
stages. Note that the rule for hacker’s movement is represented by the guards
of the form l &gt; `(s1) associated with the transitions in the net. If the hacker’s
strength level is higher than the defence level of next stage, the hacker can move
to the next stage, otherwise, the hacker can not proceed, the hacker would go
back to any previous stages.
Fig. 3. A Petri net model of a system consists of nine stages (fourteen places) and
one hacker, e0 which resides on stage s0. Note that places are represented by circles,
transitions by boxes.</p>
      <p>In general, a CPN model like that outlined above can be translated into a
behaviourally equivalent model as in (2) (essentially, each transition tr is replaced
by a set of actions obtained from the enabling bindings of tr). Similarly, each
system model as in (2) can be translated into a behaviourally equivalent CPN.
It is then possible to apply model checking tools developed for CPNs to verify
security properties in HCKC.
As mentioned in Section 4, HCKC is not only useful for analyzing attacks, but
can provide crucial information to system architects and security analysts to
make trade-off decisions involving system security. Target’s Achilles heel was the
weak security of a third-party vendor. The attacker exploited this weakness and
managed to gain access to Target’s network. In addition, another vulnerability
of Target was the failure to respond to the incident as soon as possible. Formal
modelling Target’s system in advance, using the HCKC framework security
analysts could consider that a threat can appear indirectly via third-party
interactions, according to Lateral Movement stage in the kill chain.</p>
      <p>Moreover, the exploitation of the assigned strength levels and defenders can
help the analysts to decide where to apply automated incident response techniques.
To this point, we should mention that not all adversaries are alike. Adversaries can
be individuals - insiders, outsiders, trusted/privileged insiders; groups - ad hoc
or established; organizations - competitors, suppliers, partners, customers; and
nation-state - hackers employed by a national government. Different adversaries
can aim at attacking different business processes of an organization, and these
attacks can happen concurrently. In this case a Petri net model, like the one in
Figure 3, can be extended to model these cases. We should mention that the
strength levels of entities make possible the formal verification of such complex
and highly concurrent systems.</p>
      <p>
        An alternative approach, instead of the security policies based on strength
levels, could be the adoption of weak diagnosis [
        <xref ref-type="bibr" rid="ref8">8</xref>
        ] and its associated verification
property the weakly fair diagnosability [
        <xref ref-type="bibr" rid="ref7">7</xref>
        ]. This formal verification technique
can detect abnormal behaviours of a system (e.g., cyber attacks). This approach
has been tested in [
        <xref ref-type="bibr" rid="ref34">34</xref>
        ] and showed that the detection of an adversary becomes
more “expensive" in verification time. That is, because the state space increases
dramatically when the number of entities become larger.
      </p>
      <p>Finally, it should be noted that all organizations do not have formal models
of their business processes. To employ model-based security analysis techniques
requires an investment from the organizations. Moreover, overhead costs are
incurred to maintain and optimize the formal models over time by experts.
However, the utilization of model-based security analysis techniques can provide
a return on investment as organizations squandering significant financial and
human resources on programs that are not efficient to help them diminish security
risks in their shared ecosystems.
7</p>
    </sec>
    <sec id="sec-7">
      <title>Conclusion</title>
      <p>In this paper, we developed a new framework - HCKC - to analyse how attackers
conduct the activities to organizations. Strength level of attackers and defenders
are introduced to the framework. A formal model is built to analyse the framework,
which can be captured by coloured Petri nets. This study can be used to help
security professionals find out the fundamental flaws of the security strategies of
organizations. In addition, this study can help security officers take actions to deal
with security incidents to eliminate or minimize the impact, and establish strong
controls and countermeasures in the business organizations. In the future, we
plan to produce model-based security metrics for HCKC. Thus, system architects
and Cyber Security Information Officers can use these metrics to design and
embed cost-effective security solutions to their systems.
8</p>
    </sec>
    <sec id="sec-8">
      <title>Acknowledgement</title>
      <p>The authors are very grateful to the anonymous reviewers for their detailed and
constructive comments and suggestions.</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          1.
          <string-name>
            <surname>van der Aalst</surname>
            ,
            <given-names>W.M.P.:</given-names>
          </string-name>
          <article-title>The application of petri nets to workflow management</article-title>
          .
          <source>The Journal of Circuits, Systems and Computers</source>
          <volume>8</volume>
          ,
          <fpage>21</fpage>
          -
          <lpage>66</lpage>
          (
          <year>1998</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          2.
          <string-name>
            <surname>Bryans</surname>
            ,
            <given-names>J.W.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Koutny</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Mazare</surname>
            ,
            <given-names>L.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Ryan</surname>
            ,
            <given-names>P.Y.A.</given-names>
          </string-name>
          :
          <article-title>Opacity generalised to transition systems</article-title>
          .
          <source>Int. J. Inf. Sec</source>
          .
          <volume>7</volume>
          (
          <issue>6</issue>
          ),
          <fpage>421</fpage>
          -
          <lpage>435</lpage>
          (
          <year>2008</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          3.
          <string-name>
            <surname>Bryans</surname>
            ,
            <given-names>J.W.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Koutny</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Ryan</surname>
            ,
            <given-names>P.Y.A.</given-names>
          </string-name>
          :
          <article-title>Modelling opacity using petri nets</article-title>
          .
          <source>Electr. Notes Theor. Comput. Sci</source>
          .
          <volume>121</volume>
          ,
          <fpage>101</fpage>
          -
          <lpage>115</lpage>
          (
          <year>2005</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          4.
          <string-name>
            <surname>Chen</surname>
            ,
            <given-names>T.M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Sanchez-Aarnoutse</surname>
            ,
            <given-names>J.C.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Buford</surname>
          </string-name>
          , J.:
          <article-title>Petri net modeling of cyberphysical attacks on smart grid</article-title>
          .
          <source>IEEE Transactions on Smart Grid</source>
          <volume>2</volume>
          (
          <issue>4</issue>
          ),
          <fpage>741</fpage>
          -
          <lpage>749</lpage>
          (
          <year>2011</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          5.
          <string-name>
            <surname>Death</surname>
            ,
            <given-names>D.: The</given-names>
          </string-name>
          <string-name>
            <surname>Cyber Kill Chain Explained</surname>
          </string-name>
          (
          <year>2018</year>
          ), https://www.forbes.com/ sites/forbestechcouncil/2018/10/05/the-cyber
          <article-title>-kill-chain-explained/ #1f349fd06bdf</article-title>
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          6.
          <string-name>
            <surname>Engel</surname>
          </string-name>
          , G.:
          <article-title>Deconstructing The Cyber Kill Chain (</article-title>
          <year>2014</year>
          ), https://www.darkreading. com/attacks-breaches/
          <article-title>deconstructing-the-cyber-kill-chain/a/d-id/ 1317542</article-title>
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          7.
          <string-name>
            <surname>Germanos</surname>
            ,
            <given-names>V.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Haar</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Khomenko</surname>
            ,
            <given-names>V.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Schwoon</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          :
          <article-title>Diagnosability under weak fairness</article-title>
          .
          <source>ACM Trans. Embed. Comput. Syst</source>
          .
          <volume>14</volume>
          (
          <issue>4</issue>
          ),
          <volume>69</volume>
          :
          <fpage>1</fpage>
          -
          <lpage>69</lpage>
          :
          <fpage>19</fpage>
          (
          <year>2015</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          8.
          <string-name>
            <surname>Haar</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Benveniste</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Fabre</surname>
            ,
            <given-names>E.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Jard</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          :
          <article-title>Partial order diagnosability of discrete event systems using petri net unfoldings</article-title>
          .
          <source>In: 42nd IEEE International Conference on Decision and Control</source>
          . vol.
          <volume>4</volume>
          , pp.
          <fpage>3748</fpage>
          -
          <lpage>3753</lpage>
          (
          <year>2003</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          9.
          <string-name>
            <surname>Jasiul</surname>
            ,
            <given-names>B.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Szpyrka</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Sliwa</surname>
          </string-name>
          , J.:
          <article-title>Malware behavior modeling with colored petri nets</article-title>
          .
          <source>In: Computer Information Systems and Industrial Management</source>
          . pp.
          <fpage>667</fpage>
          -
          <lpage>679</lpage>
          . Springer Berlin Heidelberg, Berlin, Heidelberg (
          <year>2014</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          10.
          <string-name>
            <surname>Kristensen</surname>
            ,
            <given-names>L.M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Simonsen</surname>
            ,
            <given-names>K.I.F.</given-names>
          </string-name>
          :
          <article-title>Applications of coloured petri nets for functional validation of protocol designs</article-title>
          .
          <source>T. Petri Nets and Other Models of Concurrency</source>
          <volume>7</volume>
          ,
          <fpage>56</fpage>
          -
          <lpage>115</lpage>
          (
          <year>2013</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          11.
          <string-name>
            <surname>Laliberte</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          :
          <string-name>
            <given-names>A</given-names>
            <surname>Twist</surname>
          </string-name>
          <article-title>On the Cyber Kill Chain: Defending Against a JavaScript Malware Attack</article-title>
          . Dark Reading (
          <year>2017</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          12. M.,
          <string-name>
            <given-names>H.E.</given-names>
            ,
            <surname>Cloppert</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.J.</given-names>
            ,
            <surname>Amin</surname>
          </string-name>
          ,
          <string-name>
            <surname>R.M.</surname>
          </string-name>
          :
          <article-title>Intelligence-driven computer network defense informed by analysis of adversary campaigns and intrusion kill chains</article-title>
          .
          <source>Bethesda</source>
          , MD: Lockheed Martin Corporation (
          <year>2010</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          13.
          <string-name>
            <surname>Marsan</surname>
            ,
            <given-names>M.A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Balbo</surname>
            ,
            <given-names>G.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Conte</surname>
            ,
            <given-names>G.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Donatelli</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Franceschinis</surname>
          </string-name>
          , G.:
          <article-title>Modelling with generalized stochastic Petri Nets</article-title>
          .
          <source>Wiley Series on Parallel Computing</source>
          (
          <year>1995</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          14.
          <string-name>
            <surname>Mason</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          :
          <article-title>Leveraging the kill chain for awesome</article-title>
          .
          <source>DARKReading</source>
          (
          <year>2014</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref15">
        <mixed-citation>
          15.
          <string-name>
            <surname>Mason</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          :
          <article-title>Leveraging the Kill Chain for Awesome (</article-title>
          <year>2014</year>
          ), https://www.darkreading.com/attacks-breaches/
          <article-title>leveraging-the-kill-chain-for-awesome/a/d-id/1317810</article-title>
        </mixed-citation>
      </ref>
      <ref id="ref16">
        <mixed-citation>
          16.
          <string-name>
            <surname>McDermott</surname>
            ,
            <given-names>J.P.</given-names>
          </string-name>
          :
          <article-title>Attack net penetration testing</article-title>
          .
          <source>In: Proceedings of the 2000 Workshop on New Security Paradigms</source>
          . pp.
          <fpage>15</fpage>
          -
          <lpage>21</lpage>
          . NSPW '00,
          <string-name>
            <surname>ACM</surname>
          </string-name>
          , New York, NY, USA (
          <year>2000</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref17">
        <mixed-citation>
          17. MITRE:
          <article-title>MITRE Research Opens Window into Cyber Attacker Behaviour</article-title>
          .
          <source>The MITRE Corporation</source>
          (
          <year>2015</year>
          ), https://www.mitre.org/news/press-releases/
          <article-title>mitre-research-opens-window-into-cyber-attacker-behavior/</article-title>
        </mixed-citation>
      </ref>
      <ref id="ref18">
        <mixed-citation>
          18.
          <string-name>
            <surname>Murata</surname>
            ,
            <given-names>T.</given-names>
          </string-name>
          :
          <article-title>Petri nets: Properties, analysis and applications</article-title>
          .
          <source>Proceedings of the IEEE</source>
          <volume>77</volume>
          (
          <issue>4</issue>
          ),
          <fpage>541</fpage>
          -
          <lpage>580</lpage>
          (
          <year>1989</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref19">
        <mixed-citation>
          19.
          <string-name>
            <surname>Myers</surname>
            ,
            <given-names>L.</given-names>
          </string-name>
          :
          <article-title>The practicality of the cyber kill chain approach to security</article-title>
          .
          <source>CSOOnline</source>
          (
          <year>2013</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref20">
        <mixed-citation>
          20.
          <string-name>
            <surname>Nachreiner</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          :
          <article-title>Kill Chain 3.0: Update the cyber kill chain for better defense (</article-title>
          <year>2015</year>
          ), https://www.helpnetsecurity.com/
          <year>2015</year>
          /02/10/ kill-chain-30
          <article-title>-update-the-cyber-kill-chain-for-better-defense/</article-title>
        </mixed-citation>
      </ref>
      <ref id="ref21">
        <mixed-citation>
          21.
          <string-name>
            <surname>Reidy</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          :
          <article-title>Combating the Insider Threat at the FBI</article-title>
          . Presented at Blackhat USA (
          <year>2013</year>
          ), https://media.blackhat.com/us-13/ US-13
          <article-title>-Reidy-Combating-the-</article-title>
          <string-name>
            <surname>Insider-Threat-At-The-</surname>
          </string-name>
          FBI-Slides.pdf
        </mixed-citation>
      </ref>
      <ref id="ref22">
        <mixed-citation>
          22.
          <string-name>
            <surname>Salimifard</surname>
            ,
            <given-names>K.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Wright</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          :
          <article-title>Petri net-based modelling of workflow systems: An overview</article-title>
          .
          <source>European Journal of Operational Research</source>
          <volume>134</volume>
          (
          <issue>3</issue>
          ),
          <fpage>664</fpage>
          -
          <lpage>676</lpage>
          (
          <year>2001</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref23">
        <mixed-citation>
          23.
          <string-name>
            <surname>Schneier</surname>
            ,
            <given-names>B.</given-names>
          </string-name>
          :
          <article-title>Secrets &amp; Lies: Digital Security in a Networked World</article-title>
          . John Wiley &amp; Sons, Inc., New York, NY, USA, 1st edn. (
          <year>2000</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref24">
        <mixed-citation>
          24.
          <string-name>
            <surname>Sheyner</surname>
            ,
            <given-names>O.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Haines</surname>
            ,
            <given-names>J.W.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Jha</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          , Lippmann,
          <string-name>
            <given-names>R.</given-names>
            ,
            <surname>Wing</surname>
          </string-name>
          ,
          <string-name>
            <surname>J.M.:</surname>
          </string-name>
          <article-title>Automated generation and analysis of attack graphs</article-title>
          .
          <source>In: 2002 IEEE Symposium on Security and Privacy</source>
          , Berkeley, California, USA, May
          <volume>12</volume>
          -15,
          <year>2002</year>
          . pp.
          <fpage>273</fpage>
          -
          <lpage>284</lpage>
          (
          <year>2002</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref25">
        <mixed-citation>
          25.
          <string-name>
            <surname>Sheyner</surname>
            ,
            <given-names>O.M.</given-names>
          </string-name>
          :
          <article-title>Scenario Graphs and Attack Graphs</article-title>
          .
          <source>PhD Thesis</source>
          , Carnegie Mellon University, (
          <year>2004</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref26">
        <mixed-citation>
          26.
          <string-name>
            <surname>Simonsen</surname>
            ,
            <given-names>K.I.F.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Kristensen</surname>
            ,
            <given-names>L.M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Kindler</surname>
          </string-name>
          , E.:
          <article-title>Pragmatics annotated coloured petri nets for protocol software generation and verification</article-title>
          .
          <source>T. Petri Nets and Other Models of Concurrency 11</source>
          ,
          <fpage>1</fpage>
          -
          <lpage>27</lpage>
          (
          <year>2016</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref27">
        <mixed-citation>
          27.
          <string-name>
            <surname>Stephenson</surname>
            ,
            <given-names>P.R.:</given-names>
          </string-name>
          <article-title>A formal model for information risk analysis using colored petri nets</article-title>
          .
          <source>In: Proceedings of the Fifth Workshop and Tutorial on Practical Use of Coloured Petri Nets and the CPN Tools</source>
          . pp.
          <fpage>167</fpage>
          -
          <lpage>184</lpage>
          (
          <year>2004</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref28">
        <mixed-citation>
          28.
          <string-name>
            <given-names>US</given-names>
            <surname>Senate</surname>
          </string-name>
          <article-title>Committee on Commerce, Science and Transportation: A 'Kill Chain' Analysis of the 2013 Target Data Breach (</article-title>
          <year>2014</year>
          ), https://www.commerce. senate.gov/public/_cache/files/24d3c229-4f2f
          <string-name>
            <surname>-</surname>
          </string-name>
          405d
          <string-name>
            <surname>-</surname>
          </string-name>
          b8db-a3a67f183883/
          <year>23E30AA955B5C00FE57CFD709621592C</year>
          .
          <fpage>2014</fpage>
          -0325
          <article-title>-target-kill-chain-analysis</article-title>
          .
          <source>pdf</source>
        </mixed-citation>
      </ref>
      <ref id="ref29">
        <mixed-citation>
          29.
          <string-name>
            <surname>Wu</surname>
            ,
            <given-names>R.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Li</surname>
            ,
            <given-names>W.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Huang</surname>
            ,
            <given-names>H.</given-names>
          </string-name>
          :
          <article-title>An attack modeling based on hierarchical colored petri nets</article-title>
          .
          <source>In: Proceedings of the 2008 International Conference on Computer and Electrical Engineering</source>
          . pp.
          <fpage>918</fpage>
          -
          <lpage>921</lpage>
          . ICCEE '08, IEEE Computer Society, Washington, DC, USA (
          <year>2008</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref30">
        <mixed-citation>
          30.
          <string-name>
            <surname>Xu</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Nygard</surname>
            ,
            <given-names>K.E.</given-names>
          </string-name>
          :
          <article-title>Threat-driven modeling and verification of secure software using aspect-oriented petri nets</article-title>
          .
          <source>IEEE Transactions on Software Engineering</source>
          <volume>32</volume>
          (
          <issue>4</issue>
          ),
          <fpage>265</fpage>
          -
          <lpage>278</lpage>
          (
          <year>2006</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref31">
        <mixed-citation>
          31.
          <string-name>
            <surname>Zeng</surname>
            ,
            <given-names>W.:</given-names>
          </string-name>
          <article-title>A methodology for cost-benefit analysis of information security technologies</article-title>
          .
          <source>Concurrency and Computation: Practice and Experience</source>
          <volume>31</volume>
          (
          <issue>4</issue>
          ),
          <year>e5004</year>
          (
          <year>2019</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref32">
        <mixed-citation>
          32.
          <string-name>
            <surname>Zeng</surname>
            ,
            <given-names>W.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Germanos</surname>
            ,
            <given-names>V.</given-names>
          </string-name>
          :
          <article-title>Benefit and cost of cloud computing security. Harnessed Causality: Essays Dedicated to Maciej Koutny on the Occasion of His 60th Birthday pp</article-title>
          .
          <fpage>143</fpage>
          -
          <lpage>150</lpage>
          (
          <year>2018</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref33">
        <mixed-citation>
          33.
          <string-name>
            <surname>Zeng</surname>
            ,
            <given-names>W.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Koutny</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          :
          <article-title>Data resources in dynamic environments</article-title>
          .
          <source>In: 2014 Theoretical Aspects of Software Engineering Conference</source>
          . pp.
          <fpage>185</fpage>
          -
          <lpage>192</lpage>
          (
          <year>2014</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref34">
        <mixed-citation>
          34.
          <string-name>
            <surname>Zeng</surname>
            ,
            <given-names>W.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Koutny</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Watson</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Germanos</surname>
            ,
            <given-names>V.</given-names>
          </string-name>
          :
          <article-title>Formal verification of secure information flow in cloud computing</article-title>
          .
          <source>J. Inf. Sec. Appl</source>
          .
          <volume>27</volume>
          -
          <issue>28</issue>
          ,
          <fpage>103</fpage>
          -
          <lpage>116</lpage>
          (
          <year>2016</year>
          )
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>