<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>Using a variety of patterns in a secure software development methodology</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Eduardo B. Fernandez</string-name>
          <email>fernande@fau.edu</email>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Nobukazu Yoshioka</string-name>
          <email>nobukazu@nii.ac.jp</email>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>Dept. of CS and EE, Florida Atlantic University</institution>
          ,
          <country country="US">USA</country>
        </aff>
        <aff id="aff1">
          <label>1</label>
          <institution>GRACE Center, Nat.Inst. of Informatics</institution>
          ,
          <country country="JP">Japan</country>
        </aff>
      </contrib-group>
      <pub-date>
        <year>2019</year>
      </pub-date>
      <fpage>26</fpage>
      <lpage>32</lpage>
      <abstract>
        <p>-  Building secure software systems requires the application of a systematic methodology. A security methodology includes a security process and a conceptual security framework consisting of security artifacts such as patterns. In this work we consider systems designed using patterns. In previous work we proposed a secure systems development methodology that uses security patterns. This methodology applies security throughout the whole lifecycle and considers all architectural levels. As part of this work we have produced a variety of security patterns. As it is difficult for designers to select security patterns, we proposed SSFs (Security Solution Frames), which are hierarchical combinations of related patterns. We introduce now a new artifact, the Security Cluster, an application-oriented combination of SSFs which further facilitates the use of security patterns to build secure applications. We also present a metamodel to get a perspective of the use of these artifacts.</p>
      </abstract>
      <kwd-group>
        <kwd>Systems security</kwd>
        <kwd>secure software development</kwd>
        <kwd>security patterns</kwd>
        <kwd>software architecture</kwd>
        <kwd>software security</kwd>
      </kwd-group>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>-</title>
      <p>INTRODUCTION</p>
      <p>
        Building a secure system requires the application of a
systematic methodology. A variety of methodologies to build
secure systems have been proposed [
        <xref ref-type="bibr" rid="ref26">27</xref>
        ]. A security
methodology includes a security process and a conceptual
security framework consisting of security artifacts such as
security patterns [
        <xref ref-type="bibr" rid="ref25 ref28">26,29</xref>
        ]. A pattern describes a solution to a
recurrent software or systems problem in a given context;
security patterns provide solutions to security problems.
Security patterns provide a way for guiding system designers
who are not experts on security to build secure systems. We
have proposed a secure systems development methodology that
uses security patterns and we have extended it recently [
        <xref ref-type="bibr" rid="ref28 ref9">9, 29</xref>
        ].
This methodology applies security throughout the whole
lifecycle and considers all architectural levels. As a
complement to this work we have produced a variety of
security patterns [
        <xref ref-type="bibr" rid="ref9">9</xref>
        ].
      </p>
      <p>Secure development methodologies apply security
solutions throughout the whole development lifecycle. A
number of those solutions have been proposed, including as
artifacts security tactics, security patterns, aspects, arguments,
formal methods, and others. We use security patterns and
related types of patterns as our artifacts; however, the number
of published security patterns is now close to 200 and it is hard
for developers to select the patterns they need in a specific
application. To help developers select patterns we proposed the
concept of Security Solution Frame (SSF) [30], which groups
together related patterns in horizontal and vertical
substructures for a single high-level policy, e.g. authentication.
We propose now a new artifact, a Security Cluster (SC), that
goes further that SSFs by gathering a set of defense
mechanisms, represented as patterns, to provide a conceptual
unit that can be added to an application to make it secure. We
provide an example of its use and we define a metamodel to
relate the new artifact to existing artifacts we have used in our
methodology to help its application. We consider the use of our
patterns in possible Architectural Knowledge Management
frameworks. The proposed SC can also be used in other
methodologies.</p>
      <p>Section II presents our view of building secure systems,
indicating that we use a model-based approach and defining
our variety of patterns. Section III describes some background.
Section IV presents the Security Cluster, our main result.
Section V considers the use of our artifacts in architecture
repositories, while Section VI is a discussion of our ideas and
describes related work. We end with conclusions in Section
VII.</p>
      <p>II.</p>
      <p>BUILDING SECURE SOFTWARE</p>
      <p>From our analysis of the literature we have identified
three basic approaches to build secure systems (Figure 1),
which include:</p>
      <p>Theoretical models—attempt to verify security
properties by using model checking but make many
assumptions which may not be true in practice and are
limited in the size of the systems that can be handled.
Cryptographic methods are a variety of theoretical
models but are effective only for specific aspects such
as system or message authentication, secure
transmission of messages, and storage protection.
They cannot stop attacks based on code or design
flaws.</p>
      <p>Code-based methods cannot find all vulnerabilities
and many attacks exploit system interactions, not code
flaws. Another problem is the complexity of code,
large systems may have tens of millions of lines of
code. Furthermore, the code changes more often than
models.
•</p>
      <p>Model-based security tries to build a strong design
structure where conceptual models describe the
different units of a system and their interactions.
Overall, the architecture of a system has a much larger
effect on security than code vulnerabilities or the
security of specific units, which makes these methods
more effective in practice. Their use of abstraction is
very valuable to handle complex systems.</p>
      <p>Theoretical</p>
      <p>Analysis of</p>
      <p>Security</p>
      <p>Model‐driven</p>
      <p>Security</p>
      <p>Formal Models 
and cryptography</p>
      <p>Code‐based</p>
      <p>Security</p>
      <p>Verification
UML/OCL models
Security patterns</p>
      <p>Certification
Vulnerability 
analysis</p>
      <p>Code 
examination
Certification
31</p>
      <p>We prefer to look for design vulnerabilities and structure a
robust architecture instead of looking for code flaws. We
believe that the best way to approach the security problem is
using models. Because of their abstraction properties, models
provide a way to apply a holistic approach to system security
and they are useful to handle large and complex systems in a
comprehensive and unified way. Of course, these approaches
are complementary, but models should define the basic
structure of any methodology. Note that the ovals in Figure 1
are not disjoint; i.e., a model-based approach may need to
consider formal proofs for critical units and code-based
analysis for heavily-used code sections.</p>
    </sec>
    <sec id="sec-2">
      <title>III. SECURE SOFTWARE</title>
      <p>.</p>
      <p>A.</p>
      <p>Applications and patterns</p>
      <p>An application is software intended to perform some
business or useful function, such as student registration,
managing accounts in a bank, and similar. An application
implements a set of use cases, where each use case is
composed of a set of activities, where each activity performs a
specific action of the use case. For example, opening an
account in a financial institution requires activities to provide
customer information, followed by a manager creating an
account and authorizations for the use of the account. These
authorizations are materialized in the form of physical cards. In
some places, an initial customer deposit is also required. Each
activity may create or use some data, which constitute assets
because they have a value for the institution. Reading or
modifying these assets are usually the attacket goals.</p>
      <p>
        The logics of the application can be described using
Analysis patterns (APs), which can be used to build a
conceptual model of the application functions. Another source
for the semantics of the application is the use of Domain
Models (DMs). A Domain Model (DM) is a conceptual model
of an area of knowledge, e.g. finance, and has no software
concepts. DMs are often described using ontologies [
        <xref ref-type="bibr" rid="ref19">19</xref>
        ] and
we can think of a DM as a compound analysis pattern
including several simpler analysis patterns that represent
specific aspects of the domain. There is a good number of
ontology patterns that describe a variety of semantic aspects in
different domains [
        <xref ref-type="bibr" rid="ref19 ref23">19, 23</xref>
        ]. Ontology patterns can be converted
into analysis patterns.
      </p>
      <p>
        A pattern is a solution to a recurrent problem in a given
context [
        <xref ref-type="bibr" rid="ref3">3</xref>
        ]. Patterns are described using a template composed
of a set of structured sections. A problem section describes a
general problem and forces that constrain and define guidelines
for the solution, e.g., “some actions must be transparent to the
users”. The solution is usually expressed using UML class,
sequence, state, and activity diagrams (although we usually
don’t need all these models). A set of consequences indicate
what is the effect of the pattern and how well the forces were
satisfied by the solution, including advantages and
disadvantages of using the pattern. An implementation section
provides hints on how to use the pattern in an application,
indicating what steps are needed and possible realizations. A
section on “Known uses” lists real systems where this solution
has been used previously, i.e., a pattern is an abstraction of
good practices. A section on related patterns indicates other
patterns that complement the pattern or that provide alternative
solutions. A pattern embodies the knowledge and experience
of software developers and can be reused in new applications;
carefully-designed patterns implicitly apply good design
principles. Patterns are also good for communication between
designers and to evaluate and reengineer existing systems.
While initially developed for software, patterns can describe
hardware, physical entities, and combinations of these, as well
as non-technical processes such as teaching a course or
organizing a conference. Pattern solutions are suggestions, not
plug-ins or software components. In particular, security
patterns can suggest solutions to designers who don’t have
much security experience. Abstract Security patterns (ASPs)
describe conceptual (no implementation aspects) security
mechanisms that realize one or more security policies able to
handle a threat or comply with a security-related regulation or
institutional policy [
        <xref ref-type="bibr" rid="ref10">10</xref>
        ]. ASPs are used in the early lifecycle
stages.
      </p>
      <p>
        The use of Reference Architectures (RAs) can simplify the
application of patterns and thus the construction and evaluation
of secure systems. A Reference Architecture (RA) is a generic
software architecture, based on one or more domains, with no
implementation aspects [
        <xref ref-type="bibr" rid="ref1">1, 25</xref>
        ]. An RA is reusable, extendable,
and configurable; that is, it is a kind of pattern for whole
architectures and it can be instantiated into a specific software
architecture by adding platform aspects [
        <xref ref-type="bibr" rid="ref1">1</xref>
        ]. We can build RAs
using patterns. After adding security patterns to neutralize
identified threats in an RA we have a Security Reference
Architecture (SRA), and we recently produced a SRA for
clouds [
        <xref ref-type="bibr" rid="ref11">11</xref>
        ]. Complete RAs or SRAs can be catalogued and
used in conjunction with appropriate development
environments.
      </p>
      <p>
        An intermediate construct between patterns, SSFs and RAs
is a Secure Semantic Analysis Pattern (SSAP) [
        <xref ref-type="bibr" rid="ref5">5</xref>
        ]. A SSAP
adds security patterns to a semantic unit based on a set of
related use cases. We have built a few of these, including legal
trials and medical records [
        <xref ref-type="bibr" rid="ref9">9</xref>
        ].
      </p>
      <p>
        We use UML for describing patterns. UML is a semiformal
language whose syntax is formally defined using a metamodel
[
        <xref ref-type="bibr" rid="ref21">21</xref>
        ]. It is widely used, many tools support its use, it is an
industry standard, and it is familiar to a wide segment of
practitioners. It can also be complemented with formal
methods, and its standard defines an associated formal
language, OCL. Being a graphic language, it is highly intuitive
and has a direct correspondence to code. There exists an
extensive literature on design and security patterns, and most
of them describe their solutions using UML.
      </p>
    </sec>
    <sec id="sec-3">
      <title>B. Threat modeling Table 1 shows examples of vulnerabilities and threats. They refer to attacks to unstructured cloud storage, such as AWS S3.</title>
      <p>
        Most methodologies, including [
        <xref ref-type="bibr" rid="ref14">14</xref>
        ], use misuse cases. We
think that misuse cases are too coarse, a use case is not atomic
but can have a good number of activities; we prefer to analyze
each activity to see how it can be subverted [
        <xref ref-type="bibr" rid="ref9">9</xref>
        ].
      </p>
      <p>
        In order to describe attacks we defined another type of
pattern: A misuse pattern describes, from the point of view of
an attacker, a generic way of performing a misuse (such as a
violation of confidentiality or integrity) that takes advantage of
the specific architecture and vulnerabilities of some
environment [
        <xref ref-type="bibr" rid="ref7">7</xref>
        ]. Until now there is only one catalog of threat
patterns [
        <xref ref-type="bibr" rid="ref27">28</xref>
        ] and several misuse patterns [
        <xref ref-type="bibr" rid="ref20 ref9">9, 20</xref>
        ], although the
concept has been studied in some detail [
        <xref ref-type="bibr" rid="ref7">7</xref>
        ]. Misuse patterns
define the environment where the attack is performed,
countermeasures to stop it, and provide forensic information in
order to trace the attack once it happens. For example, a
security defense misconfiguration is a vulnerability, taking
advantage of this vulnerability is a threat (potential attack)
which can lead to reading unauthorized information (a misuse).
In particular, misuse patterns are useful for developers because
once they determine that a possible attack can happen in the
environment, the pattern will indicate what security
mechanisms are needed as countermeasures. Also, misuse
patterns can be useful for forensic examiners to find evidence
information after the attack has been performed. Finally, they
can be used to evaluate if an existing system can handle
specific threats. Note that a misuse pattern describes a
complete attack, e.g., stealing information from a database [
        <xref ref-type="bibr" rid="ref8">8</xref>
        ],
not just specific steps used to perform the attack, such as SQL
injection or buffer overflow (both can be used in the same
attack or individually in many attacks.
      </p>
    </sec>
    <sec id="sec-4">
      <title>IV. SECURITY CLUSTERS</title>
      <p>
        We introduce here the concept of Security Cluster, starting
from Secure Solution Frames (SSFs), defined earlier. A SSF
can help a designer select patterns for all the architectural
levels of the application. We have described SSFs for
Authorization [30], Cryptography, and Authentication. SSFs
partition the solution space horizontally into Pattern Families,
which are collections of related patterns. SSFs correspond to
full realizations of security tactics [
        <xref ref-type="bibr" rid="ref2">2</xref>
        ] and can be related to
other SSFs to secure a particular application. We only consider
here SSFs that provide complementary defenses; other
relationships are discussed in [30]. We draw SSFs using
pattern diagrams [
        <xref ref-type="bibr" rid="ref3">3</xref>
        ]. A pattern diagram shows patterns as
rounded rectangles where directed edges show the contribution
of a pattern to another. Figure 2 shows a SSF for
Authentication.
      </p>
      <p>Another advantage of SSFs is that they can be used to
classify security patterns. There are now about 200 security
patterns but many are just renaming or translation of patterns to
a different architectural level. SSFs can structure patterns of
the same type by emphasizing their common concerns.</p>
      <p>Security patterns can be organized as SSFs and combined
with other artefacts. In particular, specific patterns in SSFs can
be related to specific patterns in another SSF. We can analyze
use cases to determine the threats to each application asset and
define a set of specific SSFs to control these threats, then we
can combine the corresponding SSFs.</p>
      <p>A Security Cluster (SC) is a selection of patterns from
differente SSFs. Formally: SCa= {SSFi.pa, SSFj.pb,
SSFk.pc,…}, where cluster SCa combines patterns where
SSF.pi denotes pattern i in a SSF. SCs can be catalogued by
defining a start SSF and using it as index.</p>
    </sec>
    <sec id="sec-5">
      <title>Authenticator</title>
    </sec>
    <sec id="sec-6">
      <title>Credential-based Authenticator Password-based Authenticator</title>
      <p>X.509
Certificate</p>
      <p>SAML-based</p>
      <p>Authenticator</p>
      <p>
        Fig. 3 (a set of pattern diagrams representing SSFs) shows
the construction of a Security Cluster, SC1. To define SC1 the
designer decided to use credentials as authentication artefact,
then selected Attribute-Based Access Control (ABAC), for
securing its communications she chose the Advanced
Encryption Standard, a symmetric encryption algorithm, and
finally used Distributed Logging. This specific selection was
based on the analysis of the expected threats of this application,
obtained by the method described in Section II. As we hve
shown elsewhere we can map threats to security patterns that
can stop them [
        <xref ref-type="bibr" rid="ref28 ref9">9, 29</xref>
        ].
      </p>
      <p>In a catalog, each SC description should include
recommended applications or include analysis patterns where it
would fit, as shown in Figure 3. Conversely, each security
pattern description could come with several SCs, each one
appropriate for different environments; for example, the
Account analysis pattern of Fig. 4 is appropriate for distributed,
high security environments; another version could include an
SC using passwords, RBAC, DES, and Centralized Logging.
We can also make the SCs more complex, including patterns to
protect security information, for filtering, or for secure storage.
However, in this latter case, these SCs would be less reusable.
SCs would be built by security experts and would make the
work of software developers much simpler.</p>
      <p>As indicated, we should not make the SC too specific. For
example, there are many varieties of authorization models. To
help designers choose in those cases we can use pattern
diagrams that expand parts of SCs. As an illustration of how
pattern diagrams can be used with SCs, Figure 5 shows some
variations of access control models. The double-lined patterns
show the intended selecions. The most basic access control
model is the access matrix (Basic Authorzation). This model
includes the tuple {s,o,t}, where s indicates a subject or active
entity, o is the protected object or resource, and t indicates the
type of access permitted. In that model users are allowed to
delegate their rights (discretionary property, delegatable
authorization), implying a tuple {s,o,t,f}, where f is a Boolean
copy flag indicating if the right is allowed to be delegated or
not. A predicate added to the basic rule allows content-based
authorization, becoming {s,o,t,p,f}, where p is the predicate</p>
      <sec id="sec-6-1">
        <title>Authenticator</title>
      </sec>
      <sec id="sec-6-2">
        <title>Authorizer</title>
      </sec>
      <sec id="sec-6-3">
        <title>Distributed</title>
      </sec>
      <sec id="sec-6-4">
        <title>Authenticator</title>
      </sec>
      <sec id="sec-6-5">
        <title>Security</title>
      </sec>
      <sec id="sec-6-6">
        <title>Logging SSO</title>
      </sec>
      <sec id="sec-6-7">
        <title>Credential SC1</title>
      </sec>
      <sec id="sec-6-8">
        <title>ABAC</title>
      </sec>
      <sec id="sec-6-9">
        <title>RBAC</title>
      </sec>
      <sec id="sec-6-10">
        <title>Encryption Sym AES</title>
      </sec>
      <sec id="sec-6-11">
        <title>Asym</title>
        <sec id="sec-6-11-1">
          <title>Account *</title>
          <p>owns
1</p>
        </sec>
        <sec id="sec-6-11-2">
          <title>Customer</title>
        </sec>
      </sec>
      <sec id="sec-6-12">
        <title>Distributed</title>
      </sec>
      <sec id="sec-6-13">
        <title>Logging</title>
        <p>(the predicate can also include environment variables). The
rule could also include the concept of Authorizer (a), becoming
{a,s,o,t,p,f} (Explicitly Granted Authorization). Role-Based
Access Control (RBAC) can be considered a special
interpretation of the basic authorization model, where subjects
are roles instead of individual users. Several variations and
extensions of these models have appeared. Attribute-Based
Access Control (ABAC), is the most flexible model, where any
attribute values can be used to decide access. This diagram can
be the starting point that allows a designer to select the type of
access control he needs in his application.</p>
        <p>
          Figure 6 shows a metamodel that places in perspective the
use of the different artifacts we have proposed for building a
secure application. The classes in blue describe the structure of
the application as described in Section II. The second level of
the metamodel (classes in red) describe the concepts related to
security attacks: assets have vulnerabilities that can be
exploited by attacks realizing threats. The third level (classes in
yellow), describe the countermeasures to the threats. In all
stages, classes in yellow identify artifacts that need to be
catalogued to serve as a guideline to understand threats and
apply defenses. Security patterns are realizations of policies
and tactics [
          <xref ref-type="bibr" rid="ref2">2</xref>
          ] but these are not shown in Fig. 6.
        </p>
        <p>
          To be effective all these artifacts must be organized as an
Architectural Knowledge Management (AKM) tool [
          <xref ref-type="bibr" rid="ref4">4</xref>
          ]. As
indicated earlier, security must be developed along the
lifecycle together with the functional (semantic) aspects of the
application. The application of specific patterns in an
architecture defines some of the most important design
decisions. The main uses of an AKM are: sharing of the
development activities among the stakeholders; compliance of
the design with specific quality requirements; discovery, where
the designer is helped by the tool to apply specific artifacts, and
traceability, to evaluate the impact of changes. There is now a
variety of tools [
          <xref ref-type="bibr" rid="ref4">4</xref>
          ] but in general, they do not support security
artifacts or a particular secure software development
methodology.
        </p>
        <p>
          A tool to support the security aspects of an AKM is needed.
The repository of such a tool should follow the IEEE standard
for software architecture to be compatible with existing tools.
An important use of this tool is for assurance purposes.
Assurance is a proof that a product is secure, according to
some definition of security [
          <xref ref-type="bibr" rid="ref22">22</xref>
          ]. Cyber-physical systems
require to consider also safety and reliability, which means that
more artifacts should be included in such a tool [
          <xref ref-type="bibr" rid="ref24">24</xref>
          ]. The
metamodels presented above can guide the structure of the
repository. Such a tool should also support BPMN models
which are important complements to describe requirements.
Having an explicit record of the application of artifacts in an
architecture makes this process much more systematic and
convenient.
        </p>
      </sec>
    </sec>
    <sec id="sec-7">
      <title>VI. DISCUSSION AND RELATED WORK</title>
      <p>
        As indicated earlier, security patterns encapsulate solutions
that can stop or mitigate specific threats and their consequent
misuses. This means that each pattern added to the system may
contribute to the total security of the system. However, adding
security patterns that do not stop threats would lead to systems
which are very slow, expensive, and hard to maintain. It is
clear that security patterns need a guiding methodology to be
effective and several pattern-based methodologies have been
proposed [
        <xref ref-type="bibr" rid="ref25">26</xref>
        ]; we mentioned ours earlier, a few others are
mentioned below.
      </p>
      <p>
        A general methodology for developing security-critical
software has been proposed in [
        <xref ref-type="bibr" rid="ref15">15</xref>
        ]. It makes use of an
extension of the Unified Modeling Language (UML),
UMLSec, to include security-relevant information. The
approach is supported by extensive automated tool-support for
performing a security analysis of the UMLSec models against
security requirements [
        <xref ref-type="bibr" rid="ref16">16</xref>
        ]. The analysis is based on
modelchecking specific portions of a system. This methodogy
annotates the model classes with security indications, it does
not use special artifacts.
      </p>
      <sec id="sec-7-1">
        <title>Application</title>
        <p>1..* UseCase
1..* Activity</p>
      </sec>
      <sec id="sec-7-2">
        <title>1 Threat</title>
      </sec>
      <sec id="sec-7-3">
        <title>Pattern 1</title>
        <p>*
1
realizes
1 1</p>
      </sec>
      <sec id="sec-7-4">
        <title>Attack</title>
        <p>Pattern *
realizes *</p>
      </sec>
      <sec id="sec-7-5">
        <title>1 Misuse</title>
        <p>Pattern
encapsulates *
1 Threat *
targets
*
* Asset
1
realizes
1
*
*explo1it..s* Vulnerability
1</p>
      </sec>
      <sec id="sec-7-6">
        <title>Attack</title>
        <p>encapsulates
neutralizes
* *</p>
      </sec>
      <sec id="sec-7-7">
        <title>Security</title>
      </sec>
      <sec id="sec-7-8">
        <title>Pattern (SP)</title>
        <p>1..*</p>
      </sec>
      <sec id="sec-7-9">
        <title>Compound SP</title>
      </sec>
      <sec id="sec-7-10">
        <title>Simple SP</title>
        <p>SC
* SSF
1</p>
      </sec>
      <sec id="sec-7-11">
        <title>SP Family</title>
        <p>1..*
0..1
ASP</p>
        <p>
          Mouratidis and his group use a special methodology,
Secure Tropos, to model security [
          <xref ref-type="bibr" rid="ref17">17</xref>
          ] Their work started
modeling requirements but they have also considered other
stages; for example how to test security along the lifecycle.
Instead of UML they use special diagrams and they use
patterns described in their style.
        </p>
        <p>
          Hazeyama et al. [
          <xref ref-type="bibr" rid="ref14">14</xref>
          ], include a variety of artifacts as part
of a Security Knowledge Base. They use CLASP (the
OWASP methodology) and their own methodology combined
with a knowledge base. They use security patterns, attack
patterns, and misuse cases as artifacts.
        </p>
        <p>
          Hamid [
          <xref ref-type="bibr" rid="ref13">13</xref>
          ] built a methodology and repository of
security artifacts. A metamodel for security and privacy was
presented in [
          <xref ref-type="bibr" rid="ref30">32</xref>
          ]; that model is more general than ours in
conceptual scope but less general in that it applies only to
clouds. An attempt to add extensions to UML to describe
security concerns is shown in [
          <xref ref-type="bibr" rid="ref18">18</xref>
          ], but it is mostly a notation,
it does not include new artifacts. A metamodel is used in [
          <xref ref-type="bibr" rid="ref12">12</xref>
          ]
to structure a repository for security patterns; their ontology
includes threats and attackers but its only artifacts are security
patterns. Nagaratman et al. [
          <xref ref-type="bibr" rid="ref18">18</xref>
          ] describe an AKM used by
IBM, which uses security patterns to let developers manage
development and monitor executing systems.
        </p>
        <p>In all these works there is emphasis on the general use of
the knowledge bases and on their implementation details but
not much concern about having a variety of artifacts to support
different aspects of the lifecycle or different architectural
levels. We believe a greater variety of artifacts is valuable.</p>
        <p>Examining several specific methodologies we have found
that they use few artifacts, and they could be enhanced by
using more varieties. In fact, in the final system each security
pattern will become a COTS component. Security patterns are
not intended to be coded except by the producers of the
corresponding mechanisms. Well-defined patterns make this
selection easier. In this sense, security patterns are quite
different from design patterns.</p>
        <p>Our approach is against some of the principles of agile
development processes, which emphasize producing code with
little or no use of models; while those methods are clearly
faster they are not appropriate to build secure or complex
systems.</p>
      </sec>
    </sec>
    <sec id="sec-8">
      <title>VII. CONCLUSIONS</title>
      <p>
        We introduced the concept of Security Clusters, which
appear as a good way to facilitate the work of software
developers who are not security experts. We have shown here
a variety of artifacts but they must be catalogued
systematically and stored in some knowledge base to be
effective. Although we have used a specific methodology as
example, these artifacts can be used in other methodologies
using patterns [
        <xref ref-type="bibr" rid="ref25">26</xref>
        ], and can even complement methodologies
that do not use patterns [
        <xref ref-type="bibr" rid="ref26">27</xref>
        ]. Note also that system programs
such as operating systems and database systems can also be
built using these methods; they are in fact, just specialized
applications.
      </p>
      <p>
        Applications must not only protect their data but also
comply with regulations; compliance requires systematic and
complete logging as well as access control to individual’s
information and artifact-based information makes compliance
simpler and more transparent [
        <xref ref-type="bibr" rid="ref31">33</xref>
        ].
      </p>
      <p>
        Future work will include the development of a tool
including a repository to store and manage artifacts. We have
written a partial catalog of security and misuse patterns [
        <xref ref-type="bibr" rid="ref9">9</xref>
        ] but
we need to extend it and support it through software functions.
      </p>
    </sec>
    <sec id="sec-9">
      <title>ACKNOWLEDGMENTS</title>
      <p>The work of Eduardo Fernandez was partially supported by
the National Institute of Informatics of Japan. The reviewers
provided valuable comments.  </p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          [1]
          <string-name>
            <given-names>P.</given-names>
            <surname>Avgeriou</surname>
          </string-name>
          , “
          <article-title>Describing, instantiating and evaluating a reference architecture: A case study”</article-title>
          ,
          <source>Enterprise Architecture Journal</source>
          ,
          <year>June 2003</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          [2]
          <string-name>
            <given-names>L.</given-names>
            <surname>Bass</surname>
          </string-name>
          ,
          <string-name>
            <given-names>P.</given-names>
            <surname>Clements</surname>
          </string-name>
          , and
          <string-name>
            <given-names>R.</given-names>
            <surname>Kazman</surname>
          </string-name>
          ,
          <source>Software architecture in practice (3rd Ed)</source>
          ,
          <source>Addison-Wesley</source>
          <year>2012</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          [3]
          <string-name>
            <given-names>F.</given-names>
            <surname>Buschmann</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R.</given-names>
            <surname>Meunier</surname>
          </string-name>
          ,
          <string-name>
            <given-names>H.</given-names>
            <surname>Rohnert</surname>
          </string-name>
          ,
          <string-name>
            <given-names>P.</given-names>
            <surname>Sommerlad</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Stal</surname>
          </string-name>
          .
          <source>PatternOriented Software Architecture: A System of Patterns</source>
          , Vol.
          <volume>1</volume>
          . J. Wiley,
          <year>1996</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          [4]
          <string-name>
            <given-names>R.</given-names>
            <surname>Capilla</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Jansen</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Tang</surname>
          </string-name>
          ,
          <string-name>
            <given-names>P.</given-names>
            <surname>Avgeriou</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.A.</given-names>
            <surname>Babar</surname>
          </string-name>
          , “
          <article-title>Ten years of software architecture knowledge management: Practice and future”</article-title>
          ,
          <source>The J. of Systs. and Software</source>
          ,
          <year>2016</year>
          ,
          <fpage>191</fpage>
          -
          <lpage>205</lpage>
          . DOI:
          <volume>10</volume>
          .1016/j.jss.
          <year>2015</year>
          .
          <volume>08</volume>
          .054
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          [5]
          <string-name>
            <given-names>E.B.</given-names>
            <surname>Fernandez</surname>
          </string-name>
          and
          <string-name>
            <given-names>X.</given-names>
            <surname>Yuan</surname>
          </string-name>
          , “
          <article-title>Semantic Analysis Patterns”</article-title>
          ,
          <source>Procs. 19th Int. Conf. on Conceptual Modeling, ER2000</source>
          , Salt Lake City,
          <string-name>
            <surname>UT</surname>
          </string-name>
          ,
          <source>October 2000. Lecture Notes in Computer Science</source>
          , Volume:
          <year>1920</year>
          ,
          <fpage>183</fpage>
          -
          <lpage>195</lpage>
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          [6]
          <string-name>
            <given-names>E.B.</given-names>
            <surname>Fernandez</surname>
          </string-name>
          and
          <string-name>
            <given-names>Y.</given-names>
            <surname>Liu</surname>
          </string-name>
          ,
          <article-title>"The Account Analysis Pattern"</article-title>
          ,
          <source>Procs. of EuroPLoP (Pattern Languages of Programs</source>
          ,
          <year>2002</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          [7]
          <string-name>
            <given-names>E.B.</given-names>
            <surname>Fernandez</surname>
          </string-name>
          ,
          <string-name>
            <given-names>N.</given-names>
            <surname>Yoshioka</surname>
          </string-name>
          and
          <string-name>
            <given-names>H.</given-names>
            <surname>Washizaki</surname>
          </string-name>
          ,
          <article-title>"Modeling misuse patterns"</article-title>
          ,
          <source>Procs. of the 4th Int. Workshop on Dependability Aspects of Data Warehousing and Mining Applications (DAWAM</source>
          <year>2009</year>
          ),
          <source>in conjunction with the 4th Int.Conf. on Availability</source>
          , Reliability, and
          <string-name>
            <surname>Security</surname>
          </string-name>
          (ARES
          <year>2009</year>
          ).
          <source>March 16-19</source>
          ,
          <year>2009</year>
          , Fukuoka, Japan.
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          [8]
          <string-name>
            <given-names>E</given-names>
            <surname>B. Fernandez</surname>
          </string-name>
          , E.Alder,
          <string-name>
            <given-names>R.</given-names>
            <surname>Bagley</surname>
          </string-name>
          , and
          <string-name>
            <given-names>S.</given-names>
            <surname>Paghdar</surname>
          </string-name>
          ,
          <article-title>"A Misuse Pattern for Retrieving Data from a Database Using SQL Injection"</article-title>
          , RISE'12, Workshop on Redefining and
          <article-title>Integrating Security Engineering, part of the ASE Int</article-title>
          .
          <source>Conf. on Cyber Security</source>
          , Washington, DC, December
          <volume>12</volume>
          -
          <issue>14</issue>
          ,
          <year>2012</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          [9]
          <string-name>
            <given-names>E.B.</given-names>
            <surname>Fernandez</surname>
          </string-name>
          ,
          <article-title>Security patterns in practice: Building secure architectures using software patterns”</article-title>
          .
          <source>Wiley Series on Software Design Patterns</source>
          .
          <year>2013</year>
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          [10]
          <string-name>
            <given-names>E.B.</given-names>
            <surname>Fernandez</surname>
          </string-name>
          ,
          <string-name>
            <given-names>N.</given-names>
            <surname>Yoshioka</surname>
          </string-name>
          ,
          <string-name>
            <given-names>H.</given-names>
            <surname>Washizaki</surname>
          </string-name>
          , and
          <string-name>
            <given-names>J.</given-names>
            <surname>Yoder</surname>
          </string-name>
          ,
          <article-title>"Abstract security patterns for requirements specification and analysis of secure systems'', Procs. of the WER 2014 conference, a track of the 17th IberoAmerican Conf</article-title>
          .
          <source>on Soft. Eng.(CIbSE</source>
          <year>2014</year>
          ), Pucon, Chile, April 2014
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          [11]
          <string-name>
            <given-names>E.B.</given-names>
            <surname>Fernandez</surname>
          </string-name>
          , Raul Monge, and Keiko Hashizume, “
          <article-title>Building a security reference architecture for cloud systems”</article-title>
          , Requirements Engineering. Doi:
          <volume>10</volume>
          .1007/s00766-014-0218-7,
          <year>June 2016</year>
          , Volume
          <volume>21</volume>
          , Issue 2, pp
          <fpage>225</fpage>
          -
          <lpage>249</lpage>
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          [12]
          <string-name>
            <surname>Gymnopoulos</surname>
            ,
            <given-names>L.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Karyda</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Balopoulos</surname>
            ,
            <given-names>T.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Dritsas</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Kokolakis</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Lambrinoudakis</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Gritzalis</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          :
          <article-title>Developing a Security Patterns Repository for Secure Applications De-sign</article-title>
          .
          <source>In: Proceedings of the 5th European Conf. on Information Warfare and Security (ECIW</source>
          <year>2006</year>
          ), Helsinki, Finland (
          <year>2006</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          [13]
          <string-name>
            <given-names>B.</given-names>
            <surname>Hamid</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Weber</surname>
          </string-name>
          , “
          <article-title>Engineering secure systems: Models, patterns and empirical evaluation”</article-title>
          ,
          <source>Computers &amp; Security</source>
          ,
          <volume>77</volume>
          (
          <year>2018</year>
          ),
          <fpage>315</fpage>
          -
          <lpage>348</lpage>
          . https://doi.org/10.1016/j.cose.
          <year>2018</year>
          .
          <volume>03</volume>
          .016
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          [14]
          <string-name>
            <given-names>A.</given-names>
            <surname>Hazeyama</surname>
          </string-name>
          et al., “
          <article-title>Security requirements modeling support system using software security knowledge base”</article-title>
          ,
          <source>42nd IEEE Int. Conf. on Comp. Software &amp; Applications</source>
          ,
          <year>2018</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref15">
        <mixed-citation>
          [15]
          <string-name>
            <given-names>J.</given-names>
            <surname>Jurjens</surname>
          </string-name>
          ,
          <source>Secure systems development with UML</source>
          , Springer-Verlag,
          <year>2004</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref16">
        <mixed-citation>
          [16]
          <string-name>
            <given-names>J.</given-names>
            <surname>Jurjens</surname>
          </string-name>
          , “
          <article-title>Sound methods and effective tools for model-based security engineering with UML”</article-title>
          ,
          <source>27th International Conference on Software Engineering (ICSE</source>
          <year>2005</year>
          ), ACM,
          <year>2005</year>
          , pp.
          <fpage>322</fpage>
          -
          <lpage>331</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref17">
        <mixed-citation>
          [17]
          <string-name>
            <given-names>H.</given-names>
            <surname>Mouratidis</surname>
          </string-name>
          ,
          <string-name>
            <given-names>P.</given-names>
            <surname>Giorgini</surname>
          </string-name>
          ,
          <string-name>
            <surname>G. Manson.</surname>
          </string-name>
          “
          <source>When Security Meets Software Engineering: a Case of Modelling Secure Information Systems”. Inf</source>
          . Syst., Elsevier Science Ltd., Oxford, UK, v.
          <volume>30</volume>
          , n.
          <volume>8</volume>
          , .
          <fpage>609</fpage>
          -
          <lpage>629</lpage>
          ,
          <year>2005</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref18">
        <mixed-citation>
          [18]
          <string-name>
            <given-names>N.</given-names>
            <surname>Nagaratnam</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Nadalin</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Hondo</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>McIntosh</surname>
          </string-name>
          ,
          <string-name>
            <given-names>and P.</given-names>
            <surname>Austel</surname>
          </string-name>
          , “
          <article-title>Business-driven application security: from modeling to managing secure applications”</article-title>
          ,
          <source>IBM Systems Journal</source>
          ,
          <volume>44</volume>
          (
          <issue>4</issue>
          ),
          <year>2005</year>
          ,
          <fpage>847</fpage>
          -
          <lpage>867</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref19">
        <mixed-citation>[19] http://Ontology design patterns.org</mixed-citation>
      </ref>
      <ref id="ref20">
        <mixed-citation>
          [20]
          <string-name>
            <given-names>J.</given-names>
            <surname>Pelaez</surname>
          </string-name>
          ,
          <string-name>
            <given-names>E.B.</given-names>
            <surname>Fernandez</surname>
          </string-name>
          , and
          <string-name>
            <given-names>M.M.</given-names>
            <surname>Larrondo-Petrie</surname>
          </string-name>
          ,
          <article-title>"Misuse patterns in VoIP"</article-title>
          ,
          <source>Security and Communication Networks Journal. Wiley</source>
          , Volume
          <volume>2</volume>
          ,
          <string-name>
            <surname>Issue</surname>
            <given-names>6</given-names>
          </string-name>
          , November/
          <year>December 2009</year>
          ,
          <fpage>635</fpage>
          -
          <lpage>653</lpage>
          , DOI: 10.1002/sec.105
        </mixed-citation>
      </ref>
      <ref id="ref21">
        <mixed-citation>
          [21]
          <string-name>
            <given-names>J.</given-names>
            <surname>Rumbaugh</surname>
          </string-name>
          ,
          <string-name>
            <surname>I. Jacobson</surname>
          </string-name>
          , and
          <string-name>
            <surname>G. Booch,</surname>
          </string-name>
          <article-title>The Unified Modeling Language Reference Manual</article-title>
          , Addison-Wesley, Boston, Mass.,
          <year>1999</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref22">
        <mixed-citation>
          [22]
          <string-name>
            <surname>Reijo</surname>
            <given-names>M.</given-names>
          </string-name>
          <string-name>
            <surname>Savola</surname>
          </string-name>
          <article-title>: “Quality of security metrics and measurements”</article-title>
          .
          <source>Computers &amp; Security</source>
          <volume>37</volume>
          :
          <fpage>78</fpage>
          -
          <lpage>90</lpage>
          (
          <year>2013</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref23">
        <mixed-citation>
          [23]
          <string-name>
            <given-names>L</given-names>
            <surname>Sion</surname>
          </string-name>
          ,
          <string-name>
            <given-names>K.</given-names>
            <surname>Yskout</surname>
          </string-name>
          , et al.,
          <source>“MASC: Modelling architectural security concerns”</source>
          ,
          <source>2015 IEEE/ACM 7th. Int. Workshop on Modeling in Software Engoineering”</source>
          ,
          <fpage>36</fpage>
          -
          <lpage>41</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref24">
        <mixed-citation>
          [24]
          <string-name>
            <given-names>I.</given-names>
            <surname>Sljivo</surname>
          </string-name>
          ,
          <string-name>
            <given-names>B.</given-names>
            <surname>Gallina</surname>
          </string-name>
          , “
          <article-title>Building multiple-viewpoint assurance cases using assumption/guarantee contracts”</article-title>
          ,
          <source>ISSA'16</source>
          ,
          <string-name>
            <surname>November</surname>
            <given-names>2016</given-names>
          </string-name>
          , Copenhaguen, Denmark. DOI:
          <volume>10</volume>
          .1145/1235 [251]
          <string-name>
            <given-names>R.N.</given-names>
            <surname>Taylor</surname>
          </string-name>
          ,N. Medvidovic, and
          <string-name>
            <given-names>N.</given-names>
            <surname>Dashofy</surname>
          </string-name>
          .
          <article-title>Software architecture: Foundation, theory, and practice</article-title>
          , Wiley,
          <year>2010</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref25">
        <mixed-citation>
          [26]
          <string-name>
            <given-names>A.V.</given-names>
            <surname>Uzunov</surname>
          </string-name>
          ,
          <string-name>
            <given-names>E.B.</given-names>
            <surname>Fernandez</surname>
          </string-name>
          &amp;
          <string-name>
            <surname>K. Falkner</surname>
          </string-name>
          (
          <year>2012</year>
          ),
          <article-title>"Securing distributed systems using patterns: A survey"</article-title>
          ,
          <source>Computers &amp; Security</source>
          ,
          <volume>31</volume>
          (
          <issue>5</issue>
          ),
          <fpage>681</fpage>
          -
          <lpage>703</lpage>
          . doi:
          <volume>10</volume>
          .1016/j.cose.
          <year>2012</year>
          .
          <volume>04</volume>
          .005
        </mixed-citation>
      </ref>
      <ref id="ref26">
        <mixed-citation>
          [27]
          <string-name>
            <given-names>A. V.</given-names>
            <surname>Uzunov</surname>
          </string-name>
          ,
          <string-name>
            <given-names>E.B.</given-names>
            <surname>Fernandez</surname>
          </string-name>
          , and
          <string-name>
            <given-names>K.</given-names>
            <surname>Falkner</surname>
          </string-name>
          , “
          <article-title>Engineering Security into Distributed Systems: A Survey of Methodologies”</article-title>
          ,
          <source>Journal of Universal Computer Science</source>
          , Vol.
          <volume>18</volume>
          , No.
          <volume>20</volume>
          ,
          <year>2013</year>
          , pp.
          <fpage>2920</fpage>
          -
          <lpage>3006</lpage>
          http://www.jucs.org/jucs_18_20/engineering_security_into_distributed
        </mixed-citation>
      </ref>
      <ref id="ref27">
        <mixed-citation>
          [28]
          <string-name>
            <given-names>A. V.</given-names>
            <surname>Uzunov</surname>
          </string-name>
          and
          <string-name>
            <given-names>E.B.</given-names>
            <surname>Fernandez</surname>
          </string-name>
          , “
          <article-title>An Extensible Pattern-based Library and Taxonomy of Security Threats for Distributed Systems”-</article-title>
          <source>Special Issue on Security in Information Systems of the Journal of Computer Standards &amp; Interfaces</source>
          .
          <year>2013</year>
          . http://dx.doi.org/10.1016/j.csi.
          <year>2013</year>
          .
          <volume>12</volume>
          .008
        </mixed-citation>
      </ref>
      <ref id="ref28">
        <mixed-citation>
          [29]
          <string-name>
            <given-names>Anton</given-names>
            <surname>Uzunov</surname>
          </string-name>
          , E. B Fernandez, Katrina Falkner, “
          <article-title>ASE: A Comprehensive Pattern- Driven Security Methodology for Distributed Systems”</article-title>
          ,
          <source>J. of Comp. Standards &amp; Interfaces</source>
          , Vol.
          <volume>41</volume>
          ,
          <string-name>
            <surname>Sept</surname>
          </string-name>
          .
          <year>2015</year>
          ,
          <fpage>112</fpage>
          -
          <lpage>137</lpage>
          [30]
          <string-name>
            <given-names>Anton</given-names>
            <surname>Uzunov</surname>
          </string-name>
          , E. B Fernandez, Katrina Falkner, “
          <article-title>Security solution frames and security patterns for authorization in distributed, collaborative systems”</article-title>
          ,
          <source>Computers &amp; Security</source>
          ,
          <volume>55</volume>
          ,
          <year>2015</year>
          , pp.
          <fpage>193</fpage>
          -
          <lpage>234</lpage>
          , doi: 10.1016/j.cose.
          <year>2015</year>
          .
          <volume>08</volume>
          .003
        </mixed-citation>
      </ref>
      <ref id="ref29">
        <mixed-citation>
          [31]
          <string-name>
            <surname>Anton</surname>
            <given-names>V.</given-names>
          </string-name>
          <string-name>
            <surname>Uzunov</surname>
          </string-name>
          , Eduardo B.
          <string-name>
            <surname>Fernandez</surname>
          </string-name>
          , Katrina Falkner, “
          <article-title>Assessing and Improving the Quality of Security Methodologies for Distributed Systems”, accepted for the</article-title>
          <source>Journal of Software: Evolution and Process</source>
        </mixed-citation>
      </ref>
      <ref id="ref30">
        <mixed-citation>
          [32]
          <string-name>
            <given-names>H.</given-names>
            <surname>Washizaki</surname>
          </string-name>
          et al. ,
          <article-title>“A metamodel for security and privacy knowledge in cloud services</article-title>
          ”,
          <source>2016 IEEE World Congress on Services (SERVICES) DOI: 10.1109/SERVICES</source>
          .
          <year>2016</year>
          .30
        </mixed-citation>
      </ref>
      <ref id="ref31">
        <mixed-citation>
          [33]
          <string-name>
            <given-names>D.</given-names>
            <surname>Yimam</surname>
          </string-name>
          and
          <string-name>
            <given-names>E. B.</given-names>
            <surname>Fernandez</surname>
          </string-name>
          , “
          <article-title>Building Compliance and Security Reference Architectures for Cloud Systems”</article-title>
          ,
          <source>IEEE Int. Conf. On Cloud Engineering (IC2E)</source>
          <year>2016</year>
          , Berlin, April 4-
          <issue>8</issue>
          ,
          <year>2016</year>
          .
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>