<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>Security (hyper-)properties in workflow systems: From specification to verification?</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Thomas Bauereiss Advisor: Dieter Hutter</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>German Research Center for Artificial Intelligence (DFKI) Bremen</institution>
          ,
          <country country="DE">Germany</country>
        </aff>
      </contrib-group>
      <abstract>
        <p>specification towards an implementation. We focus our attention on workflow management systems due to their interesting security requirements and the widespread use of model-driven techniques in this area (e.g. using BPMN diagrams). We build upon existing verification techniques for a specific notion of information flow security, and intend to apply our results to concrete example systems such as a secure web-based conference management system.</p>
      </abstract>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>-</title>
      <p>As computer systems grow increasingly complex and pervade more and more
aspects of everyday life, reliable security properties become increasingly important.
In large, distributed systems that facilitate the collaboration of multiple users
there are different types of security requirements that need to be satisfied by the
subsystems. The confidentiality and integrity of data items that are processed in
the system needs to be protected, and there are security requirements regarding
the users involved in the process, e.g. separation of duty constraints, requiring
that at least two users must agree on a joint decision before the corresponding
action can be taken.</p>
      <p>Addressing these security requirements already in early phases of the
development process avoids costly changes to the architecture and design of the
system in later phases. Tool support for (semi-)automatic analysis of security
aspects is needed to cope with the increasing complexity of computer systems.
This analysis should be based on well-founded models and theories of computer
security in order to allow reliable guarantees of security properties.
? This research is supported by the Deutsche Forschungsgemeinschaft (DFG) under
grant Hu737/5-1, which is part of the DFG priority programme 1496 “Reliably Secure
Software Systems.”</p>
      <p>
        Existing approaches to workflow security typically map security requirements
to access control configurations (e.g. [
        <xref ref-type="bibr" rid="ref22 ref5">5, 22</xref>
        ]), while some formalise security
requirements as LTL formulas and employ model-checking for verification (e.g. [
        <xref ref-type="bibr" rid="ref19 ref2">2,
19</xref>
        ]). This is suitable for safety or liveness properties, but it is insufficient for
many notions of information flow security that can be seen as hyperproperties
[
        <xref ref-type="bibr" rid="ref7">7</xref>
        ]. Information flow control goes beyond mere access control by taking into
account the complete behaviour of the system, thereby preventing not only direct
but also implicit information leaks via observation of the system. Formally, while
safety and liveness properties correspond to sets of traces (i.e. system runs),
hyperproperties have been defined in [
        <xref ref-type="bibr" rid="ref7">7</xref>
        ] as sets of sets of traces, or equivalently,
as predicates on complete systems. In particular, possibilistic information flow
predicates can be seen as closure predicates on trace sets [
        <xref ref-type="bibr" rid="ref14">14</xref>
        ], e.g. requiring that
for each trace with a confidential event, an alternative trace without the event
must exist that yields the same visible observations.
      </p>
      <p>Enforcing a safety property by removing traces that violate the property
can potentially destroy information flow security: For example, consider a secure
workflow system where an additional separation of duty constraint between a
confidential and a non-confidential activity is to be enforced. Someone who can
observe the non-confidential activity and sees a certain user perform it can
deduce that this user has not participated in the confidential activity. This might
be an information leak in itself (if anonymity is a concern), and if different users
are allowed to perform different actions it might even leak information about
the exact actions that could have been performed in the confidential activity.
Hence, there can be subtle interrelations between different types of security
requirements, and our aim is to develop a framework where they can be treated
in an integrated way and refined from the specification to the implementation
level.
2</p>
    </sec>
    <sec id="sec-2">
      <title>Aims and Objectives</title>
      <p>
        We focus on workflow management systems due to their interesting security
requirements. In [
        <xref ref-type="bibr" rid="ref4">4</xref>
        ], we used a hiring workflow as a running example, where
medical data about applicants has to be kept confidential and separation of duty
constraints between different medical officers have to be enforced. A developer
who wants to implement such a system needs to map the security requirements
to a secure implementation in some way. Our vision is a step-wise development
process, starting from high-level specifications of the system (e.g. a BPMN
workflow diagram) and its security requirements (e.g. as annotations in the workflow
diagram) that are mapped to a formal model. Refinement techniques and tools
then support the developer in performing refinement steps towards an
implementation in such a way that security properties established on the abstract level are
preserved by the refinement. In such a refinement step, the developer can replace
an activity in a workflow by a subprocess, or refine behavioural specifications
of atomic activities. We want this development process to be well-founded on
formal models and theories of security so that the resulting implemented system
has provable security properties. Our goal is to build upon existing formal
verification techniques and identify and close existing gaps along the way from an
initial workflow specification to a verified implementation. In particular, we aim
at the following contributions:
– A framework for specifying workflow systems and their relevant security
requirements: Workflow security is typically understood in terms of access
control, with some exceptions [
        <xref ref-type="bibr" rid="ref1 ref23">23, 1</xref>
        ]. However, [
        <xref ref-type="bibr" rid="ref23">23</xref>
        ] does not state the
actual information flow property that it checks in a declarative,
mechanismindependent way, while [
        <xref ref-type="bibr" rid="ref1">1</xref>
        ] focuses on a specific notion of information flow
that can be checked by a structural analysis of a Petri net representation of
the workflow. We choose to build upon the MAKS framework for
possibilistic information flow [
        <xref ref-type="bibr" rid="ref14">14</xref>
        ], as it unifies several existing notions of information
flow from the literature. We model workflows as state-event systems
suitable for verification in the MAKS framework, and map data requirements to
information flow predicates and process requirements to safety properties.
– A verification framework for both data and process requirements: We adapt
an existing methodology for compositional verification of information flow
security, and we propose to use a compositional approach to verify the
compatibility of information flow predicates and safety properties [
        <xref ref-type="bibr" rid="ref3">3</xref>
        ]. We found
that this leads to intuitive and reusable results in the cases that we have
considered, and we believe it can be a useful complement to existing approaches
to security-preserving refinement, e.g. [
        <xref ref-type="bibr" rid="ref15">15</xref>
        ].
– An improved technique for action refinement in MAKS [
        <xref ref-type="bibr" rid="ref10">10</xref>
        ] in order to move
from a specification closer to the implementation level: [
        <xref ref-type="bibr" rid="ref10">10</xref>
        ] allows to
replace atomic abstract events with sequences of more concrete events, and
gives sufficient conditions for the preservation of security by such a
refinement. However, it requires configuration structure semantics in order to deal
with concurrency, and the conditions for preservation of security are rather
strict. We see room for improvement wrt. relaxing these conditions, possibly
integrating insights from related approaches for other formalisms [
        <xref ref-type="bibr" rid="ref17 ref21 ref6">6, 17, 21</xref>
        ].
      </p>
      <p>
        On a more technical level, we develop the above techniques not only using
pen and paper, but also within the interactive theorem prover Isabelle/HOL
based on an existing formalisation of the MAKS framework. This serves to
verify our results and also to connect to other formalisations and tools available
for Isabelle/HOL, e.g. the large HOL library for the specification of software in
terms of functional programs, formalisations of operational semantics of some
programming languages, e.g. [
        <xref ref-type="bibr" rid="ref12">12</xref>
        ], tool support for data refinement [
        <xref ref-type="bibr" rid="ref13">13</xref>
        ], or code
generation from specifications to languages such as Scala [
        <xref ref-type="bibr" rid="ref9">9</xref>
        ]. Ideally, our
formalised theories will become the foundation of an analysis tool for workflow
security that can be integrated into a workflow modelling tool.
3
      </p>
    </sec>
    <sec id="sec-3">
      <title>Research plan</title>
      <p>Our research is on formally specifying and verifying security properties of
software systems, hence the research methodology centers around the formalisation
of the system and desired properties, and the development of proof techniques
for verification. Our work done so far has focused on modelling and verifying
security at the abstract level, so future work will focus on the aspects of refinement,
tool support, and evaluation.
3.1</p>
      <sec id="sec-3-1">
        <title>Work done to date</title>
        <p>
          System model: In [
          <xref ref-type="bibr" rid="ref4">4</xref>
          ], we have formalised a workflow management system on an
abstract level in terms of a composition of subsystems representing the activities
in the workflow. This facilitates the distributed deployment of workflow systems,
e.g. using web services, and for compositional verification of security. Activities
in the workflow are mapped to state-event systems that pass on data items and
control flow triggers by sending messages to each other. The design decisions
regarding the interactions of these activities were inspired by the BPMN
standard, which describes the execution semantics of the control flow, for example, in
terms of tokens that are passed from one activity to the next one, corresponding
to trigger messages in our model. Essentially, our model captures a basic subset
of BPMN, and it should be straightforward to add more complex aspects such
as exception handling.
        </p>
        <p>
          Security properties: In [
          <xref ref-type="bibr" rid="ref4">4</xref>
          ], we have mapped data requirements to information
flow predicates. System events representing the input of a confidential data item
are classified as confidential events, and the events belonging to activities with
a lower security classification as visible. The information flow predicates then
formalise the requirement that someone who observes or participates in visible
activities cannot deduce information about the occurence or non-occurence of
confidential events and, hence, the values of confidential data items.
        </p>
        <p>We model process requirements such as separation of duty as safety
properties, characterising the sets of execution traces that do not violate the security
requirement. This allows us to formally model such security requirements on the
abstract level without having to refer to implementation details of the
enforcement mechanism.</p>
        <p>
          Compositional verification: For verification of information flow security, we apply
a decomposition methodology [
          <xref ref-type="bibr" rid="ref11">11</xref>
          ] to split up the overall security requirement
into requirements for the individual components and then prove that they satisfy
these requirements using an unwinding technique.
        </p>
        <p>
          We propose to use compositionality also for the integrated verification of
information flow security and safety properties [
          <xref ref-type="bibr" rid="ref3">3</xref>
          ]. A safety property can be
enforced using an execution monitor that runs in parallel with the target system
and inhibits or modifies executions that would violate the safety property [
          <xref ref-type="bibr" rid="ref20">20</xref>
          ].
We can analyse such a monitor and verify that it does not leak confidential
information under certain conditions, and then compose it with the target system.
The composed system satisfies the safety property, and the compositionality
theorems of the MAKS framework give us sufficient conditions under which this
composition preserves information flow security. We demonstrate this approach
for separation of duty and for ordered delivery of messages in [
          <xref ref-type="bibr" rid="ref3">3</xref>
          ].
3.2
        </p>
      </sec>
      <sec id="sec-3-2">
        <title>Future work</title>
        <p>
          Refinement: This work so far only takes into account one single level of
abstraction. As discussed above, action refinement can be used to map from one level
of abstraction to a more concrete one while retaining the security properties
established on the abstract level. Eventually, we want to reach the implementation
level using such a refinement. This means relating abstract traces to concrete
program executions as well as abstract values to concrete data structures, e.g.
XML documents. Relating an abstract specification and an implementation in a
concrete programming language requires a formal semantics of the language in
terms of execution traces, and, preferably, a security type system for establishing
the security of programs. There are semantics for realistic languages formalised
in Isabelle, e.g. a subset of Java [
          <xref ref-type="bibr" rid="ref12">12</xref>
          ]. For a simple while-language, a relation
between MAKS security predicates and a language-based notion of information
flow security that can be checked using a security type system has been
established in [
          <xref ref-type="bibr" rid="ref16">16</xref>
          ].
        </p>
        <p>
          An alternative approach to is to generate an executable implementation from
a sufficiently detailed specification using a (trusted) code generator. A potential
target language is Scala, as Isabelle supports code generation [
          <xref ref-type="bibr" rid="ref9">9</xref>
          ] for it, and the
Akka library available for Scala could be used to implement the activities in a
workflow system as actors in an actor system.
        </p>
        <p>
          We will investigate these approaches, formalise a suitable refinement
technique in Isabelle, and integrate it into our workflow formalisation.
Evaluation: We aim to evaluate our results in example scenarios. Within the
scope of the DFG priority programme “Reliably Secure Software Systems” we
currently collaborate with other research projects on a joint reference scenario
on Web-based workflow management systems. The concrete example application
is a Web-based conference management system, with security requirements such
as confidentiality of submissions, anonymity of reviewers, and separation of duty
between reviewers and authors. We intend to evaluate our techniques in this
scenario and hope to benefit from the collaboration with the other projects, working
on aspects such as model-checking information flow security [
          <xref ref-type="bibr" rid="ref8">8</xref>
          ] or language-based
noninterference [
          <xref ref-type="bibr" rid="ref18">18</xref>
          ].
        </p>
        <p>Another demonstrator could be the integration of our techniques into an
open-source business process modeling tool, so that a developer can model a
workflow, annotate it with security requirements, add or refine behavioural
specifications or code to activities in the workflow, and generate an executable
implementation. On the basis of our formal semantics for workflows and the
verification techniques we employ, such a tool could alert developers to security
problems already during specification and throughout the refinement process,
guiding them to a provably secure workflow system.</p>
        <p>In the long term, we hope that our work on the formal foundations of workflow
security and tools building upon them will contribute to increased scalability and
adoption of formal methods for the engineering of workflow systems with strong,
provable security guarantees.</p>
      </sec>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          1.
          <string-name>
            <surname>Accorsi</surname>
            ,
            <given-names>R.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Lehmann</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          :
          <article-title>Automatic information flow analysis of business process models</article-title>
          .
          <source>In: BPM. LNCS</source>
          , vol.
          <volume>7481</volume>
          , pp.
          <fpage>172</fpage>
          -
          <lpage>187</lpage>
          . Springer (
          <year>2012</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          2.
          <string-name>
            <surname>Arsac</surname>
            ,
            <given-names>W.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Compagna</surname>
            ,
            <given-names>L.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Pellegrino</surname>
            ,
            <given-names>G.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Ponta</surname>
            ,
            <given-names>S.E.</given-names>
          </string-name>
          :
          <article-title>Security validation of business processes via model-checking</article-title>
          . In: ESSoS. No. 6542
          <string-name>
            <surname>in</surname>
            <given-names>LNCS</given-names>
          </string-name>
          , Springer (
          <year>2011</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          3.
          <string-name>
            <surname>Bauereiss</surname>
            ,
            <given-names>T.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Hutter</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          :
          <article-title>Compatibility of safety properties and possibilistic information flow security in MAKS</article-title>
          .
          <source>In: Proc. IFIP SEC</source>
          <year>2014</year>
          , to appear (
          <year>2014</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          4.
          <string-name>
            <surname>Bauereiss</surname>
            ,
            <given-names>T.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Hutter</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          :
          <article-title>Possibilistic information flow security of workflow management systems</article-title>
          . In: GraMSec'14, to appear in EPTCS (
          <year>2014</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          5.
          <string-name>
            <surname>Bertino</surname>
            ,
            <given-names>E.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Ferrari</surname>
            ,
            <given-names>E.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Atluri</surname>
            ,
            <given-names>V.</given-names>
          </string-name>
          :
          <article-title>The specification and enforcement of authorization constraints in workflow management systems</article-title>
          .
          <source>ACM Trans. Inf. Syst. Secur</source>
          .
          <volume>2</volume>
          (
          <issue>1</issue>
          ),
          <fpage>65</fpage>
          -
          <lpage>104</lpage>
          (
          <year>Feb 1999</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          6.
          <string-name>
            <surname>Bossi</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Piazza</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Rossi</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          :
          <article-title>Action refinement in process algebra and security issues</article-title>
          .
          <source>In: LOPSTR</source>
          <year>2007</year>
          , pp.
          <fpage>201</fpage>
          -
          <lpage>217</lpage>
          . No. 4915
          <string-name>
            <surname>in</surname>
            <given-names>LNCS</given-names>
          </string-name>
          , Springer (
          <year>2008</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          7.
          <string-name>
            <surname>Clarkson</surname>
            ,
            <given-names>M.R.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Schneider</surname>
            ,
            <given-names>F.B.</given-names>
          </string-name>
          : Hyperproperties.
          <source>Journal of Computer Security</source>
          <volume>18</volume>
          (
          <issue>6</issue>
          ),
          <fpage>1157</fpage>
          -
          <lpage>1210</lpage>
          (
          <year>2010</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          8.
          <string-name>
            <surname>Dimitrova</surname>
            ,
            <given-names>R.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Finkbeiner</surname>
            ,
            <given-names>B.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Kovács</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Rabe</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Seidl</surname>
          </string-name>
          , H.:
          <article-title>Model checking information flow in reactive systems</article-title>
          . In: VMCAI (
          <year>2012</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          9.
          <string-name>
            <surname>Haftmann</surname>
            ,
            <given-names>F.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Nipkow</surname>
            ,
            <given-names>T.</given-names>
          </string-name>
          :
          <article-title>A code generator framework for Isabelle/HOL</article-title>
          . In: Theorem Proving in Higher Order Logics: Emerging Trends (
          <year>2007</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          10.
          <string-name>
            <surname>Hutter</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          :
          <article-title>Possibilistic information flow control in MAKS and action refinement</article-title>
          .
          <source>In: ETRICS</source>
          , pp.
          <fpage>268</fpage>
          -
          <lpage>281</lpage>
          . No. 3995
          <string-name>
            <surname>in</surname>
            <given-names>LNCS</given-names>
          </string-name>
          , Springer (
          <year>2006</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          11.
          <string-name>
            <surname>Hutter</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Mantel</surname>
            ,
            <given-names>H.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Schaefer</surname>
            ,
            <given-names>I.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Schairer</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          :
          <article-title>Security of multi-agent systems: A case study on comparison shopping</article-title>
          .
          <source>J. Applied Logic</source>
          <volume>5</volume>
          (
          <issue>2</issue>
          ),
          <fpage>303</fpage>
          -
          <lpage>332</lpage>
          (
          <year>Jun 2007</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          12.
          <string-name>
            <surname>Klein</surname>
            ,
            <given-names>G.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Nipkow</surname>
            ,
            <given-names>T.:</given-names>
          </string-name>
          <article-title>A machine-checked model for a java-like language, virtual machine, and compiler</article-title>
          .
          <source>ACM Trans. Program. Lang. Syst</source>
          .
          <volume>28</volume>
          (
          <issue>4</issue>
          ),
          <fpage>619</fpage>
          -
          <lpage>695</lpage>
          (
          <year>2006</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          13.
          <string-name>
            <surname>Lammich</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          :
          <article-title>Automatic data refinement</article-title>
          .
          <source>In: Interactive Theorem Proving</source>
          , pp.
          <fpage>84</fpage>
          -
          <lpage>99</lpage>
          . No. 7998
          <string-name>
            <surname>in</surname>
            <given-names>LNCS</given-names>
          </string-name>
          , Springer (Jan
          <year>2013</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          14.
          <string-name>
            <surname>Mantel</surname>
          </string-name>
          , H.:
          <article-title>Possibilistic definitions of security-an assembly kit</article-title>
          .
          <source>In: CSFW</source>
          (
          <year>2000</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref15">
        <mixed-citation>
          15.
          <string-name>
            <surname>Mantel</surname>
          </string-name>
          , H.:
          <article-title>Preserving information flow properties under refinement</article-title>
          .
          <source>In: IEEE Security &amp; Privacy</source>
          . pp.
          <fpage>78</fpage>
          -
          <lpage>91</lpage>
          . IEEE (
          <year>2001</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref16">
        <mixed-citation>
          16.
          <string-name>
            <surname>Mantel</surname>
            ,
            <given-names>H.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Sabelfeld</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          :
          <article-title>A unifying approach to the security of distributed and multi-threaded programs</article-title>
          .
          <source>Journal of Computer Security</source>
          <volume>11</volume>
          (
          <issue>4</issue>
          ),
          <fpage>615</fpage>
          -
          <lpage>676</lpage>
          (
          <year>2003</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref17">
        <mixed-citation>
          17.
          <string-name>
            <surname>Martinelli</surname>
            ,
            <given-names>F.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Matteucci</surname>
            ,
            <given-names>I.</given-names>
          </string-name>
          :
          <article-title>Preserving security properties under refinement</article-title>
          .
          <source>In: SESS</source>
          . pp.
          <fpage>15</fpage>
          --
          <lpage>21</lpage>
          . ACM (
          <year>2011</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref18">
        <mixed-citation>
          18.
          <string-name>
            <surname>Popescu</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Hölzl</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Nipkow</surname>
            ,
            <given-names>T.</given-names>
          </string-name>
          :
          <article-title>Formal verification of language-based concurrent noninterference</article-title>
          .
          <source>Journal of Formalized Reasoning</source>
          <volume>6</volume>
          (
          <issue>1</issue>
          ),
          <fpage>1</fpage>
          -
          <lpage>30</lpage>
          (
          <year>2013</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref19">
        <mixed-citation>
          19.
          <string-name>
            <surname>Schaad</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Lotz</surname>
            ,
            <given-names>V.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Sohr</surname>
            ,
            <given-names>K.</given-names>
          </string-name>
          :
          <article-title>A model-checking approach to analysing organisational controls in a loan origination process</article-title>
          .
          <source>In: SACMAT'06</source>
          .
          <string-name>
            <surname>ACM</surname>
          </string-name>
          (
          <year>2006</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref20">
        <mixed-citation>
          20.
          <string-name>
            <surname>Schneider</surname>
            ,
            <given-names>F.B.</given-names>
          </string-name>
          :
          <article-title>Enforceable security policies</article-title>
          .
          <source>ACM Trans. Inf. Syst. Secur</source>
          .
          <volume>3</volume>
          (
          <issue>1</issue>
          ),
          <fpage>30</fpage>
          -
          <lpage>50</lpage>
          (
          <year>Feb 2000</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref21">
        <mixed-citation>
          21.
          <string-name>
            <surname>Seehusen</surname>
            ,
            <given-names>F.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Stølen</surname>
            ,
            <given-names>K.</given-names>
          </string-name>
          :
          <article-title>Maintaining information flow security under refinement and transformation</article-title>
          .
          <source>In: FAST 2007. LNCS 4691</source>
          , pp.
          <fpage>143</fpage>
          -
          <lpage>157</lpage>
          . Springer (
          <year>2007</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref22">
        <mixed-citation>
          22.
          <string-name>
            <surname>Wolter</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Menzel</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Schaad</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Miseldine</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Meinel</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          :
          <article-title>Model-driven business process security requirement specification</article-title>
          .
          <source>J. Syst. Architect</source>
          .
          <volume>55</volume>
          (
          <issue>4</issue>
          ),
          <fpage>211</fpage>
          -
          <lpage>223</lpage>
          (
          <year>2009</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref23">
        <mixed-citation>
          23.
          <string-name>
            <surname>Yang</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Lu</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Gofman</surname>
            ,
            <given-names>M.I.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Yang</surname>
            ,
            <given-names>Z.</given-names>
          </string-name>
          :
          <article-title>Information flow analysis of scientific workflows</article-title>
          .
          <source>Journal of Computer and System Sciences</source>
          <volume>76</volume>
          (
          <issue>6</issue>
          ),
          <fpage>390</fpage>
          -
          <lpage>402</lpage>
          (
          <year>Sep 2010</year>
          )
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>