<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta>
      <journal-title-group>
        <journal-title>Requirement</journal-title>
      </journal-title-group>
    </journal-meta>
    <article-meta>
      <title-group>
        <article-title>Cloud Native Simulation of Reference Nets</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>University of Hamburg</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Faculty of Mathematics</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Informatics</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Natural Sciences</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>Department of Informatics</institution>
        </aff>
      </contrib-group>
      <volume>1</volume>
      <issue>2</issue>
      <fpage>85</fpage>
      <lpage>104</lpage>
      <abstract>
        <p>This contribution presents the integration of the cloud-native paradigm into the distributed simulation of reference nets. This is done to reap the bene ts of cloud-based environments and by that to increase the scaling capabilities of such simulations on an architectural level. In contrary to most Petri net formalisms, reference net simulations are able to allocate net instances at runtime, like objects are instantiated in object oriented programming. Therefore, reference nets are not static in size and well suitable to natively model dynamic systems with changing demands and net structures. During the architectural discussion the key concepts of operability, observability, agility, and resilience are addressed. Further, a proof-of-concept implementation illustrates the potential of such cloudbased systems. It utilizes the simulator Renew and the Java Spring framework.</p>
      </abstract>
      <kwd-group>
        <kwd>High-level Petri nets</kwd>
        <kwd>Cloud infrastructure</kwd>
        <kwd>Software Architecture</kwd>
        <kwd>Distributed Simulation</kwd>
        <kwd>Renew</kwd>
        <kwd>Containerization</kwd>
      </kwd-group>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>-</title>
      <p>The interaction of large sets of agents in hierarchical structures is omnipresent in
the real-world. While humans are certainly agents in this notion, machines and
organizational entities can be as well. Through simulation of these kinds of
interactions, information about real-world scenarios can be deduced. In the context
of reference nets, a high-level Petri net formalism, that supports concepts
similar to object-oriented programming, several publications regarding simulation
of hierarchical agents exist.</p>
      <p>While several important questions have been addressed by these publications,
the simulations usually ran on a single physical machine and by that, we are
restricted in its size. While the simulation of traditional Petri nets is usually
static in size with the exception of token count in some cases, the size of a
reference net simulation cannot be determined at compile time. This is due
to their concept of net instances, which may be created dynamically during
runtime. Therefore, scalability is an interesting issue to be solved in the context
Copyright © 2021 for this paper by its authors. Use permitted under Creative
Commons License Attribution 4.0 International (CC BY 4.0).
of simulating reference nets. It addresses the ability to dynamically extend the
simulation to more physical notes in an automated and unsupervised fashion.</p>
      <p>
        There has been research to execute a reference net simulation over
multiple machines using the Distribute plugin for the simulator Renew [
        <xref ref-type="bibr" rid="ref18">18</xref>
        ].
Subsequently, virtualization [
        <xref ref-type="bibr" rid="ref13">13</xref>
        ] and container technology [
        <xref ref-type="bibr" rid="ref17">17</xref>
        ] have been used to
further abstract from speci c infrastructure and enable higher portability of
Renew simulators. The control over the simulation extension has been
incorporated into the simulation itself to allow for proactive scaling of instances in
[
        <xref ref-type="bibr" rid="ref16">16</xref>
        ]. While the aforementioned publications only address the core functionality
to distribute the simulation and the control of the environment, there are no
considerations regarding the software architecture of the simulator in regards to
scalability.
      </p>
      <p>Cloud nativity is a paradigm to construct applications, that do not rely on
the possibility for operators to be able to access the systems they are running on.
With increasing independence on the underlying infrastructure, the application
is able to spread to additional physical nodes more easily.</p>
      <p>Cloud native applications are also popular in the community of microservice
applications. These are usually run in cloud environments, that are constructed
and destructed dynamically and applications are deployed automatically.</p>
      <p>This paper contributes software architecture related solutions to incorporate
the paradigm of cloud nativity into reference net simulators, especially the
Renew simulator. Our foreseen bene ts are a more exible use of the simulator,
abstraction from speci c protocols like e.g. Java RMI, and better incorporation
into scalable environments. Our contribution is focused on the backend-part of
the simulator. Therefore any graphical representations or consumers are referred
to and used for motivation in the following sections, but are not part of the core
contribution. As an exception, we use the ready-made user interface of Spring
Boot Admin1 in the evaluation part of this contribution.</p>
      <p>In chapter 2 we explain the basics. The general strategy on how to approach
an integration is covered by chapter 3. Following that, chapter 4 describes the
conceptual integration of Cloud Native Renew. In chapter 5 we elucidate our
implementation. Lastly, chapter 6 serves as an evaluation of our work, and
chapter 7 represents a nal conclusion.
1.1</p>
    </sec>
    <sec id="sec-2">
      <title>Motivation</title>
      <p>When attempting the simulation of large-scale systems scalability of the
simulating software components is bene cial. Using a paradigm such as cloud nativity
can directly improve this aspect of software. Without dependence on system
access to the infrastructure running the simulation, as well as robustness
towards other components and agility of the software, many manual steps can be
necessary to incorporate additional physical nodes into the simulation.</p>
      <sec id="sec-2-1">
        <title>1 https://github.com/codecentric/spring-boot-admin - All URLs have been accessed</title>
        <p>in March 2021</p>
        <p>The requirement for the introduction of scalable reference nets originated
from the construction of dynamic multi-agent multi-platform systems. The
overall approach is far too large to su ciently summarize here. The gist of the
approach is the introduction of a platform management system, that is able to
proactively summon heterogeneous reference net based agent platforms into
existence and deploy speci c functionality to it, as well as to move agents between
these kind of platforms.</p>
        <p>Looking from the Petri net point of view, there should be no di culty in
ring concurrent transitions on physically distant nodes. However, due to the
complexity of the simulation of systems described with reference nets, this can be
di cult to achieve. A cloud native simulator also favors more universal protocols
for information retrieval, such as HTTP, and by that enables the simulation to
be accessible in di erent ways from di erent software and systems.</p>
        <p>With a universal interface to the software, it is e.g. possible to easily split
simulation and representation of the simulation. Additionally, a cloud native
simulator could be used without installing it on a local system rst, enabling
even low spec clients like e.g. netbooks to run complex simulations easily. Also,
it is possible to interact with the simulation without a local Java environment
installation.</p>
        <p>Another useful factor is the ability for the simulator to report other states of
the simulator, which would otherwise be di cult to determine from the system's
point of view, that is running the simulator. Examples for these states are the
number of simulated net instances, loaded plugins of the simulator, liveliness
of the simulated reference net, and so on. These could be accessed in the same
fashion as external attributes such as CPU load or memory consumption.
1.2</p>
      </sec>
    </sec>
    <sec id="sec-3">
      <title>Related Work</title>
      <p>
        Several research papers have been published concerning the integration of Petri
nets and service-oriented structures, like [
        <xref ref-type="bibr" rid="ref12">12</xref>
        ], that considers the performance
of web services but focuses on business processes and service allocation. In a
similar fashion, [
        <xref ref-type="bibr" rid="ref1">1</xref>
        ] considers performance issues on services but aims mainly at
mathematical models. Other publications use di erent Petri net subformalisms
to interact with services, like e.g. [
        <xref ref-type="bibr" rid="ref5">5</xref>
        ] or [
        <xref ref-type="bibr" rid="ref2">2</xref>
        ].
      </p>
      <p>
        The combination of container technology and Petri nets has been discussed
in [
        <xref ref-type="bibr" rid="ref3">3</xref>
        ] and [
        <xref ref-type="bibr" rid="ref17">17</xref>
        ]. Implementation-wise [
        <xref ref-type="bibr" rid="ref15">15</xref>
        ] introduces rst ideas regarding the
integration of the Spring framework with reference nets.
2
      </p>
      <sec id="sec-3-1">
        <title>Basics</title>
        <p>
          Introducing the topic, we now explain the core concepts reference nets, Renew,
and cloud nativity.
Reference nets are colored Petri nets that convey the object-oriented mindset
and combine the concept of nets-within-nets [
          <xref ref-type="bibr" rid="ref19">19</xref>
          ] with communication via
synchronous channels. Reference nets feature the concept of net instances, that
correspond to objects in object-oriented programming, while nets (or "net
definitions") correspond to classes. Like colored nets with arbitrary color sets,
reference nets are Turing-complete. Also, setting up net hierarchies is possible by
referencing speci c instances. Synchronous channels synchronize multiple
transitions to re simultaneously. This way, it allows multidirectional information
passing via uni cation across multiple net instances. The transitions must agree
on the channel name and the set of parameters in order to establish the
synchronization.
2.2
        </p>
      </sec>
    </sec>
    <sec id="sec-4">
      <title>Renew</title>
      <p>
        Renew [
        <xref ref-type="bibr" rid="ref10">10</xref>
        ] is a multi-formalism editor and simulator developed by the
Department of Informatics of the University of Hamburg. It o ers a exible modeling
approach for reference nets via a user-friendly graphical interface. Renew's key
values, openness, and versatility have various reasons. Firstly, it runs on Java,
and therefore on almost every single operating system. It can also make use of
any Java class by adding them to a transition inscription within a reference net.
Lastly, since reference nets are also Java objects, the communication between
the code and the reference nets is straightforward. Renew is controlled by a
plugin system [
        <xref ref-type="bibr" rid="ref4">4</xref>
        ] which o ers a high degree of decoupling. Renew is available
for download on the projects' website2.
2.3
      </p>
    </sec>
    <sec id="sec-5">
      <title>Renew Remote</title>
      <p>
        Renew features a "remoting" layer between the simulation core and the user
interface. It is based on Java RMI3 and has been introduced in Renew 1.6 in
[
        <xref ref-type="bibr" rid="ref11">11</xref>
        ]. As it is currently still the most advanced feature of Renew to provide
operability and observability, it is used as a point of reference in this paper. Like
all Java RMI-based implementations do, Renew Remoting requires an RMI
registry to keep track of remote objects.
      </p>
      <p>Upon starting a remote-enabled simulation, a second instance of Renew may
connect to the running simulation over RMI. It is then possible to control the
simulation from the second instance as if it was running locally. This includes
starting, stopping, resuming, stepping, and terminating a simulation, as well as
ring transitions manually. It is also possible to view instantiated net instances,
as well as the current marking and transition rings. Renew Remoting, however,
is limited to a connection to a single simulator.</p>
      <sec id="sec-5-1">
        <title>2 http://www.renew.de</title>
      </sec>
      <sec id="sec-5-2">
        <title>3 Java RMI is the Java implementation of the Remote Method Invocation protocol</title>
      </sec>
    </sec>
    <sec id="sec-6">
      <title>Renew Distribute Plugin</title>
      <p>
        The Distribute plugin was introduced by Michael Simon [
        <xref ref-type="bibr" rid="ref18">18</xref>
        ]. It allows the
construction of distributed simulations in Renew. The plugin is also based on
the Remote Method Invocation protocol and by that on Java RMI.
      </p>
      <p>
        It alters the core of the simulation algorithm in various ways. The details
are less important for the considerations in this contribution and are therefore
omitted. The publication [
        <xref ref-type="bibr" rid="ref18">18</xref>
        ] shows the inner workings of the plugin in more
detail. Implementation-wise the Distribute algorithm corresponds to a limited
extent to a classic distributed commit protocol.
2.5
      </p>
    </sec>
    <sec id="sec-7">
      <title>Modular Renew</title>
      <p>A recent contribution to the Renew simulator is the support of modularity. It is
employed using the Java Jigsaw project, which was released with Java 9. Its main
bene ts are improved encapsulation within the application by using explicitly
de ned interfaces between modules. Module dependencies are also de ned as a
directed acyclic graph instead of arbitrary connections.</p>
      <p>As an additional step, modular Renew also uses so-called module layers,
which provide a means to apply existing module principles to code, that is loaded
at runtime. Module layers also provide the base for the ability to dynamically
unload plugins, which is interesting in the context of hot-swapping
implementations within a running application. By now, each Renew plugin resides in its
own module, which is loaded inside a separate layer.</p>
      <p>Modular Renew can also be downloaded from the project's website as
"preview of Renew 4.0".
2.6</p>
    </sec>
    <sec id="sec-8">
      <title>Cloud Nativity</title>
      <p>Cloud nativity is an approach in software development of growing importance,
which utilizes cloud computing to create scalable applications in dynamic
environments.</p>
      <p>
        Cloud nativity's strength is to even function on an unknown system's
infrastructure without the need for operators to manually access it. There are several
de nitions of cloud nativity. As it ts our intended architecture the best, we will
refer to the one presented in [
        <xref ref-type="bibr" rid="ref8">8</xref>
        ]. According to Garrison and Nova, cloud nativity
has the four key bene ts resiliency, agility, operability, and observability.
      </p>
      <p>Firstly, a cloud native system is resilient, when the failure of one component
of the system doesn't cause other components to fail also. This is really valuable,
as a chain reaction could take down the entire system.</p>
      <p>The system is also agile, as quick changes according to new requirements can
easily be made. A part of this capability is rooted in the architecture: the system
consists of microservices and uses containerization for deployment. Containers
are used to package all software needed into one executable package and should
be provided using a continuous integration pipeline. This allows the project to
be independent of the environment. Also, agile development approaches are used
to enable quick reactions to changing demand.</p>
      <p>The application should publish interfaces to administrate the application and
by that provide operability. Using them, an administrator can perform con
guration tasks or administration of users, without the need to rely on underlying
systems.</p>
      <p>And lastly, observability is given by the provision of information about
ongoing processes, for example, health metrics and log data.
3</p>
      <sec id="sec-8-1">
        <title>Approaching an integration</title>
        <p>To realize the integration of cloud nativity with reference net simulations, we
rst outline the shortcomings in the current architecture of Renew and its
relevant plugins in regards to cloud nativity. We do so by covering the four
aspects of observability, operability, agility, and resilience separately. To achieve
a successful integration, the addressed problems need to be solved. From within
each aspect, we will construct speci c requirements to be met.</p>
        <p>Observability is mainly realized by the Remote plugin. The run of a single
simulation can be observed through an additional client and the Java RMI
protocol. This includes live transitions rings, markings, and an event log. System
information and meta-information about the simulator are not included.
Consuming the information requires an additional Renew simulator, which again
requires an underlying Java virtual machine. Certain situations in a remote
simulation, like deadlocks, can be observed manually by an observer knowledgeable
in Petri nets, but not automatically.</p>
        <p>We can now formulate the requirements for the integration:
1. The rst and most important goal is to introduce a universal interface for the
aforementioned functionalities, that can be used without the need to rely on
a local Java installation and Renew software. This also means to decouple
it from graphical user interface components.
2. Observability should cover the entire distributed simulation, not just singular
parts of it.
3. The environment (system resources) of a remote simulator should be
observable through the interface.</p>
        <p>Operability is also mainly realized by the Remote plugin. Upon attaching
to a remote simulation the simulation may be halted, stepped, terminated, etc.
This, however, is also only possible with a local installation of Renew and over
Java RMI. To a degree, the Distribute plugin could also be used to "operate"
a remote simulation by ring certain distributed synchronous (send) channels.
The operation, however, is only implicit as it is only limited to the inner
workings of the simulation itself. Operability was also not the intended use of the
Distribute plugin and it would require an additional local Renew simulation.</p>
        <p>Requirements regarding operability are therefore:
4. Operability also requires a more exible interface. This interface should be
identical to the one from observability.
5. The simulator should be extensible by new net de nitions and not just net
instances. Modi cation of existing net de nitions on the other hand
potentially leads to non-trivial and large complications and therefore will not be
addressed in this context.
6. The simulator should be extensible with additional functionality, that might
be required to run speci c net de nitions.</p>
        <p>
          Agility is available in several aspects of Renew. In regards to agile
development, Renew is developed using the Scrum@Scale approach4. It is also
integrated with extensive continuous integration support and by that using
automated pipelines for building. Concerning portability, the publication [
          <xref ref-type="bibr" rid="ref13">13</xref>
          ]
examined virtualization-based deployment and [
          <xref ref-type="bibr" rid="ref17">17</xref>
          ] presented deployments based
on containerization. In uence on the execution infrastructure is possible for
simulations with proactive scaling, as introduced in [
          <xref ref-type="bibr" rid="ref16">16</xref>
          ]. Agility aspects are well
covered already, therefore there is not much left to be done in this regard.
        </p>
        <p>As the aspects are well covered through other works in the context of the
Renew simulator, we can only express one requirement:
7. The simulator needs to utilize all the features referenced in the "Agility"
section simultaneously.</p>
        <p>Resilience is mainly relevant in regards to the distribution and inclusion
of external services. The Distribute plugin is built on top of Java RMI and
therefore requires a single RMI registry for all nodes in the overall deployment.
The registry needs to be present during the start of each simulator participating
in the simulation. It also must not disconnect during the simulation and net
instances of past runs might still be known to the registry leading to
inconsistencies. These problems can be alleviated by restarting the registry before an
actual simulation, though. Further, relying on external services is not generally
addressed on the simulator implementation level. This is due to the fact, that
the ecosystem was more or less closed for external services so far. But
introducing a exible and universal interface, as motivated earlier, opens the possibility
to incorporate other external (web)services into the simulation runtime. These
services should not be addressed directly, to avoid cascading failures or local
corruption.</p>
        <p>Overall we were able to de ne the following requirements for the resilience
of a cloud native reference net simulation:
8. Failures of global system components must not lead to cascading failures.
9. Failures of local nodes must not endanger the overall simulation.
10. Simulation processes must not directly rely on external services, the structure
of their delivered data, or on the quality of their delivered data.</p>
        <sec id="sec-8-1-1">
          <title>4 see https://www.scrumatscale.com/</title>
        </sec>
      </sec>
      <sec id="sec-8-2">
        <title>Conceptual Integration</title>
        <p>In this section we will present the concepts we propose to address the
requirements explained in the last section. We believe, that these su ce to achieve a
thorough integration of cloud nativity and the reference net simulator Renew.
The aspects observability, operability, and resilience will again be addressed one
by one. As agility is already well covered, it will not be addressed separately in
the conceptual section. An overview of the utilized tools and methods will follow
in section 5, which will present our prototype.</p>
        <p>We will design the overall system with the global architecture displayed in
gure 1 in mind. Several aspects and necessities are only elaborated on further
into this section, however, it will be helpful to deliver the bigger picture early on
to explain some aspects more easily. Note, that speci c infrastructure elements,
like physical nodes, are omitted in this diagram, as the goal of the combination of
cloud nativity and the Renew simulator is an infrastructure-agnostic simulation
environment in the rst place.</p>
        <p>The gure shows several separate services, that are required or at least
recommended in the deployment of cloud native Renew. Note, that this paper only
discusses the cloud native Renew plugin, which is shown in bold font. However,
to understand certain implications and design choices it is helpful to be able to
see the bigger picture. Elements, that are drawn on top of each other, like for
example the cloud native Renew plugin and the Renew simulation core are
two parts of the same service, while separate elements are separate services. Not
showing the infrastructure also implies, that these services can each run on a
di erent machine or all together on one single machine or something in between.</p>
        <p>Some of the prominent parts are the several Renew simulation cores, that
are shown in green, alongside their orange colored plugins. There are just two
Renew simulation cores depicted in the gure due to tidiness. As scalability is
one of the claims for this contribution, it is intended to be run with much more
than just two instances of Renew. The three dots on the right-hand side of the
gure also indicate this.</p>
        <p>As the cloud native Renew plugin is designed to o er an interface to
incorporate external services, two of them have been drawn into the gure as
an example. Note, that this is for future-proofness, as there are no
incorporated external services as of now, as also mentioned in the approach section. For
convenient access to all operability and observability aspects, the central
pinkcolored control panel can be used. There is no speci c implementation of this
for reference nets, but as we will show within the evaluation section, depending
on the way the aspects of operability and observability are introduced into the
simulation, ready-made tool suites can be used.</p>
        <p>
          Further, one of the additional aspects is the distributed message broker shown
in light blue. It acts as the central message passing system in the deployment and
is mainly used by an updated Distribute Renew plugin. Additionally, there
are some special external services displayed and labeled as consumers. These are
able to extract information recorded by the Renew simulation cores from the
message broker and consume them in several individual ways. An example for a
consumer is a web-based GUI, like the one introduced for reference nets by [
          <xref ref-type="bibr" rid="ref9">9</xref>
          ].
The consumer of said publication, however, was not wired to a simulation so far,
because of the missing link, that is also lled within the context of cloud native
Renew.
        </p>
        <p>Finally, at the bottom of the gure "Renew Petri net Saga executor(s)"
are shown in light green. These are intended for eventual consistency in the
distributed ring process and are described later on in more detail in section 4.3
about the introduction of resilience into the system.
4.1</p>
      </sec>
    </sec>
    <sec id="sec-9">
      <title>Introducing Observability</title>
      <p>The rst aspect to introduce is observability. Using Renew remote it is possible
to use the Renew simulator to connect to a remotely running simulation. This is
done using the Java RMI protocol and requires a local Renew simulator. While
this solution provides excellent observability of a single simulation it does not
provide a exible interface. This means, that some kind of Java environment is
required to consume the interface. It also does not provide means to check on
loaded plugins and system states of the remote simulation. Renew remote is
also tightly coupled to the graphical user interface of Renew, which again relies
on the existence of some kind of output screen, and by that, it might encounter
unforeseen di culties in non-graphical container-based deployments.</p>
      <p>To acquire a su cient degree of observability a universal interface is required
to access speci c information. We argue that HTTP is a good choice in this
regard, as it is in fact universally employed and wide-spread, stateless, and su
cient in terms of expressiveness. All relevant system information, that is required
by the control panel should be available. This includes memory consumption,
available persistent space on the system, and processor load. As processor load
might experience spiking during the invocation of the request to the information
HTTP endpoint it is also desirable to be able to request average load within the
last seconds. Also, general information on the healthiness of the process is of
interest to be able to decide, whether the speci c simulation core might require
restarting or even manual attention.</p>
      <p>The application should present these kinds of health metrics within a single
or multiple HTTP endpoints.</p>
      <p>Reference net simulations on Renew can di er from each other in regard
to the loaded plugins by the simulator. Incorporating an additional simulation
node into a running simulation will only be successful if the node in question is
aware of all used classes and is capable of the required functionality. It is not
explicitly necessary to match the plugin pro le of the remaining simulation cores,
but the simulator should provide a minimum baseline of capabilities to ful ll its
role in the simulation. Therefore currently loaded plugins should be announced
upon start on the message bus and also should be available as information to
the control panel. Certainly, the plugins themselves might hold data, that is of
interest like the core simulator plugin holds information over liveliness and net
instance count of the simulation. A uni ed interface for plugins to supply this
information is required for the internal passing of information from the plugins
to the HTTP endpoint.</p>
      <p>Finally, to be able to trace certain error states of the simulator, it would
be desirable to acquire the stdout logging output of the application. Almost
every infrastructure solution nowadays can supply these from the infrastructure
itself, but to stay within the requirements of being totally independent on the
underlying infrastructure, the simulator should also be able to supply its stdout
log upon request.</p>
      <p>The last and largest part to support observability of the simulation addresses
the access to ring events of a running simulation. The remote plugin supplies
this only for one simulator, but a global transparent simulation feed of the overall
simulation is more desirable. Construction of a simulation feed for reference
nets is not easy though. Due to the nature of the tokens, that, in fact, may be
references to arbitrary Java objects, describing them is non-trivial.</p>
      <p>As displayed in gure 1, displaying the global architecture, a central event
log system will be available for external consumers and inter-simulation
communication. This event log will store information about the simulation and will be
fed by each individual simulator. Consumers then can access the log and display
the simulation e.g. in a user interface or consume the process otherwise, like
waiting for a certain transition ring.
4.2</p>
    </sec>
    <sec id="sec-10">
      <title>Introducing Operability</title>
      <p>As outlined in the approach section, the Remote plugin for Renew supports
basic operability with the Java RMI protocol. In the last section, we have shown,
that a web-based interface over the HTTP protocol will bene t the application
in terms of exibility and requirements on the clients. Therefore we need to
redesign the operability presented from within the Renew Remote plugin to
match the new interface. A basic operability interface needs to at least provide
means to start, stop, step, resume and terminate a simulation.</p>
      <p>Further, in the context of service-based deployments, it will be desirable to
extend a running service without the need to shut it down rst. Also, the supply
of net de nitions to run should be possible easy. Renew utilizes a construct
called shadow net systems. A shadow net is a net de nition, that is stripped from
all of its information, that is non-critical for actual execution. Single or multiple
shadow nets can form a shadow net system, which is a set of these. Shadow nets
also o er the ability to run net compilers on nets to prepare them for execution.
They also use binary serialization, when saved in contrary to regular reference
nets, which use a text-based solution and are by that larger in size.</p>
      <p>As a shadow net is independent of its graphical representation and holds all
information to construct a net instance from it to execute, it perfectly ts for the
supplying of running reference net simulator services with new net de nitions. It
is also possible to verify a shadow net system at least for basic validity by parsing
the shadow net system object structure, which we will do. The system should also
be able to distinguish between submitting simulation candidates using shadow
net systems and commands to actually instantiate a new simulation.</p>
      <p>Net de nitions alone can o er the possibility to run the arbitrary workload
on a remote and distributed reference net simulator. Reference nets are
Turingcomplete and able to express any programmable functionality. However, often
functionality can be reused, which is done in the shape of plugins for the
simulator. Renew plugins can be compiled to jar les and can be loaded by the
loader plugin, which is the central component of Renew.</p>
      <p>An interface providing operability should be able to accept additional
plugins, as well as loading them into the simulation code. We are aware of the
security implications of this behavior, as a potential attacker could just upload
malware and execute it most easily. This also applies to the upload of net de
nitions, as they are Turing-complete, as stated before. As for now, however, we
do not address this aspect in this contribution and assume all involved parties
are trustworthy. For future versions, there is currently undergoing research and
implementation work is done in our workgroup regarding net de nition signing
and plugin signing prior to loading to tackle these problems.</p>
      <p>While loading a plugin (or generally speaking a jar library) is straightforward
in Java, unloading them is not. Therefore we need to assume, that supplied
classes are not yet existent under the speci ed name (classpath). Unloading
might become possible using the Java module system, which was outlined earlier.</p>
      <p>
        Another aspect of operability is the in uence on the simulation environment.
As this has already been addressed in the publication [
        <xref ref-type="bibr" rid="ref16">16</xref>
        ] in the shape of
RenewKube, we will omit this aspect here and assume infrastructure interaction
is possible and available.
4.3
      </p>
    </sec>
    <sec id="sec-11">
      <title>Introducing Resilience</title>
      <p>One of the major concerns regarding resilience is the avoidance of single points
of failure and the avoidance of cascading failures. As outlined earlier, the
Distribute plugin is one problematic part, especially the used RMI registry and
RMI calls. The left part of Table 1 shows what parts of the algorithms in the
Distribute plugin use which approaches.</p>
      <p>Approach so far</p>
      <p>Algorithm part</p>
      <p>New approach</p>
      <p>Registry
Direct via RMI call
local</p>
      <p>Net instance referencing</p>
      <p>Firing</p>
      <p>Binding
Uni cation</p>
      <p>Message broker
Direct via HTTP
local</p>
      <p>We argue, that both, RMI registry and RMI calls can be replaced by using a
stateless protocol like HTTP and a persistent message broker system, which is
replicated (sharded). Stateless protocols demand much less constrained
environments than Java RMI does. It also can be inspected more easily for debugging
and monitoring considerations.</p>
      <p>
        With the introduction of a message broker system, the referencing to remote
instances can be modeled just as with an RMI registry. The advantages are,
however, that using general message brokers can be accessed more exibly and
do not require a speci c running Java application. It also persists the net
instance information and does not require ongoing communication to the service
hosting the remote net instance. To di erentiate between separate simulations,
we propose to apply a UUID5 identi er to each simulation upon start. The
identi er is then passed to new simulation participants using the infrastructure (see
RenewKube [
        <xref ref-type="bibr" rid="ref16">16</xref>
        ]), which is not addressed here. To alleviate possible failures
of the message broker, a replicated and sharded system like e.g. Apache Kafka6
should be used.
      </p>
      <p>Upon retrieval of remote net instances, the distributed binding search should
be triggered over HTTP by directly contacting the Renew instance, that the
net instance is hosted on. This can be done in an easy fashion, as binding search
is a read-only operation 7. When a binding is found it can be red. The
Distribute plugin uses RMI calls for this and by freezing the related parts, as
described earlier, it e ectively handles distributed transition ring in a similar
manner, as a distributed commit protocol. This behavior, however, introduces
the possibility for cascading failures, or more likely non-responsiveness of several</p>
      <sec id="sec-11-1">
        <title>5 Universally unique identi er: These can be generated locally and global collisions</title>
        <p>are as low as 1 in 2122 and by that negligible.</p>
      </sec>
      <sec id="sec-11-2">
        <title>6 https://kafka.apache.org/</title>
      </sec>
      <sec id="sec-11-3">
        <title>7 Note, that the internal implementation of the binding search in Renew is com</title>
        <p>plex and requires evaluation of possible bindings. It uses so-called state recorders to
e ectively create an overall read-only operation.
system components, once a badly timed outage happens during distributed
ring. Therefore we argue to introduce an approach featuring eventual consistency
and availability orientation.</p>
        <p>
          One of these approaches is the Saga pattern [
          <xref ref-type="bibr" rid="ref7">7</xref>
          ], which rather recently found
its revival in the context of microservices [
          <xref ref-type="bibr" rid="ref14">14</xref>
          ]. We can utilize Petri net Sagas
to implement distributed transition ring, which is discussed in the separate
publication "Petri Net Sagas" within these proceedings of PNSE 2021.
        </p>
        <p>This overall concept covers the challenges introduced by the Distribute
plugin but external services are yet to cover. Upon relying on external services,
the central processes of the simulator should never directly rely on the
availability of the service or the integrity of the delivered data. Therefore we argue
to introduce a convention to apply so-called Anti-Corruption-Layer (ACLs) in
front of external services. These encapsulate the interface towards the service
and behave in a deterministic and foreseeable way towards the core
implementations. The bene cial aspects of this are stronger encapsulation of the external
behavior and resilience against potentially malicious behavior of the service.
5</p>
        <sec id="sec-11-3-1">
          <title>Implementation</title>
          <p>As outlined in the conceptual section, the implementation of the cloud nativity
aspects requires the introduction of new functionality, as well as adjustments to
existing functionality. Adjustments mainly address the Distribute plugin and
resilience considerations along with integration problems with message broker
systems. The implementation of both would exceed the scope of a single
paper. Therefore, we only present an implementation for the introduction of new
functionality, mainly regarding observability and operability. These are also the
aspects, that serve as a foundation for the introduction of resilience later on.</p>
          <p>Since the implementation of a web service is quite complex, we decided to
use the Spring framework for our prototype. The procedure and problems that
occurred, as well as their solutions, are described in the following sections.
5.1</p>
        </sec>
      </sec>
    </sec>
    <sec id="sec-12">
      <title>Spring Framework and Dependency Injection</title>
      <p>Java Spring is a framework that is often used for web applications. In addition,
Spring Boot is a variant of the framework that allows a "ready to run" version,
with minimal features for web functionality, to be set up quickly. This further
reduces the already shortened implementation time. Besides the faster
implementation time, it is also much easier to integrate a Spring context into Renew
than to implement all elements needed for a minimal web application. More on
that in the "Integration with Renew " section. Because of the minimal version of
Spring we are using, we can easily integrate all the features that Spring provides
and add only the functionality we need. A currently used extension is Spring
Boot Admin, which provides a control panel and an interface for health metrics.
Other interesting extensions o ered by the framework include database
integration and Object-Relation-mapping through Hibernate, as well as an integrated
cloud architecture.</p>
      <p>
        Java Spring also allows us to use the concept of dependency injection. This
concept gives the possibility of providing a kind of blueprint for objects of a
class that we can then pass to the required location. If an object of this speci c
class is needed, one can be inserted by injection of the required object [
        <xref ref-type="bibr" rid="ref6">6</xref>
        ]. The
framework provides automated initialization of objects by accessing the
initialization de nition provided by the developer of the corresponding class. Thus,
users of an object of the class do not need to know how to initialize that
object. We have chosen the form of constructor injection for the implementation
because it allows us to see how and where objects are used and initialized easier.
In addition, the use of dependency injection ensures that we achieve an increase
in the decoupling of the components, which is in line with the concept of cloud
nativity.
5.2
      </p>
    </sec>
    <sec id="sec-13">
      <title>Integration with Renew</title>
      <p>As mentioned before we used the Java Spring framework which we integrated
into Renew. Figure 2 shows that we put Spring in a separate module layer to
create a system where Spring is integrated in Renew. This way we can guarantee
that all Spring functionalities that do not support a module system can work
without any problems.</p>
      <p>In our attempt to integrate Spring with Renew, we created a new plugin and
added a new Spring Boot project to it. After some adjustments to the module
properties according to the Renew speci cations, we were able to identify two
core problems.</p>
      <p>Primarily all Spring modules and libraries were not present in the project.
The problem comes from the fact that the Java module system is not thoroughly
integrated into Spring and this con icts with the modular order given by Renew.
Furthermore, Spring Boot creates a fat jar that contains all dependencies and
necessary libraries if Spring is a standalone application. However, the Renew
loader can do little with it. To integrate the contents in the build process of
Renew we made adjustments to the location settings, where we were able to
place the contents of the fat jar separately where Renew organizes the modules
and libraries.</p>
      <p>The other problem occurs after deploying the required modules and libraries,
Spring does not have access to them. Due to the module layer structure, the
dependencies for Spring and our module are on di erent layers, and thus cannot
have access to each other. Our solution was to implement a new module layer
only for Spring and its required modules, as shown in gure 2.</p>
      <p>Since for many internal Renew operations, certain outputs or errors are
issued for certain behavior, we had to introduce a system in which we can react
to various internal events. Generally, we use the HTTP status codes, as well as a
JSON response with more detailed information as return value for each request.
We obtained this by implementing our own response class that provides reliable
responses independent of the internal Renew process.</p>
      <p>To ful ll observability aspects, we implemented a log output and a status
overview as a health metric. For the implementation of the log output, we
redirected the two Java output streams System.out and System.err, and formatted
the contents of the streams for the respective endpoints. The health metrics are
provided by Spring Actuator and include simple information about the system
where Spring, i.e. Renew, is running on. After some additions to this given
implementation, it is now possible to receive information like memory, CPU usage,
RAM usage, and their corresponding averages about the host system.
Furthermore, we also provide an overview of all loaded Renew plugins and created
another plugin that acts as a communicator between all other Renew plugins
and Spring. This new plugin gives all components the ability to display a
message in the health metric. We decided to create a new plugin for this because
if we had provided this functionality in our cloud native layer, there could be a
circular dependency on plugins in the module system. Another solution would
have been to integrate it directly into the Spring plugin, but then Spring would
need to know the implementation details of the plugins it uses. However, both
other approaches are in contradiction to the architecture of Renew.</p>
      <p>In consideration of the operability aspect, we have created a possibility to
upload reference nets, as well as a way to control these nets. When uploading the
nets, we accept a le and check whether it is the correct le format by attempting
to parse its shadow net structure. If this is the case, the le is deposited in the
appropriate path. We also implemented an endpoint to upload additional Renew
plugins, which then can be loaded into the running simulator at runtime. As
outlined earlier, we do not check for malware yet in both cases. For the control
of the simulation, we currently provide four commands: "run" for starting a
simulation, "step" for stepping a simulation, "stop" for stopping a simulation,
and "term" for terminating a simulation. Here we used the given functionality of
the simulator plugin in Renew and mapped them to the corresponding HTTP
endpoint. A key problem is that at the time of receiving the request and sending a
corresponding response, only the submission of the command into the simulator
can be guaranteed, due to the multi-threaded and concurrent behavior of the
simulation core of Renew.
6</p>
      <sec id="sec-13-1">
        <title>Evaluation</title>
        <p>For evaluation purposes, the implementation was run on a physical machine
running Windows 10 20H2 with 32 GB memory and an AMD Ryzen 9 3900X
12-core CPU. The speci c evaluation criteria introduced in section 3 were used.
As motivated earlier, we have split the implementation between the cloud native
Renew plugin and the upgrades to the Distribute plugin. As the latter is
addressed in another upcoming contribution, the evaluation also only covers the
aspects realized within the cloud native Renew plugin itself. The
implementation of the cloud native Renew plugin can also be downloaded from the projects
website8. An overview of the ful lled requirements can be found in table 2.
9</p>
        <p>10
Ful lled?</p>
        <p>X X X X X X X (X) (X) (X)</p>
        <p>With the implementation, the simulation can be run on a server. It is
possible to use the endpoint "/simulation/start" to start a simulation over HTTP.
It is not required to have a local installation of Renew or Java to do so, but it
is still necessary to have any HTTP client installed. We thereby meet
requirement 1. The simulator also supports control of the simulation via the respective
endpoints. Commands to control the simulation can be submitted via HTTP,
which ful lls requirement 4. Further, it is possible to upload new net de
nitions as shadow net system, which subsequently can be instantiated with the
start simulation endpoint, ful lling requirement 5. Additionally, Renew plugins
can be uploaded and loaded during runtime through the web interface, ful lling
requirement 6.</p>
        <p>Health metrics can indicate the correct functioning of the simulator. We
enriched these with environment information (meeting requirement 3) and a means
for di erent parts of the application to present custom status data. As each
simulator implements these endpoints, we can use a centralized monitoring system
to access all states simultaneously, ful lling requirement 2. For our evaluation,
we ran the tool-suite Spring Boot Admin to consume the endpoints emitted by
the simulators. The tool-suite corresponds to the "Control Panel" in gure 1. A
view of the user interface of Spring Boot Admin for our simulator is visible in
gure 3.</p>
        <sec id="sec-13-1-1">
          <title>8 https://paose.informatik.uni-hamburg.de/paose/wiki/CloudNativeRenew</title>
          <p>
            The evaluation of the agility aspects of requirement 7 is not straightforward,
as it requires other dependent projects. Mainly the contribution presented in [
            <xref ref-type="bibr" rid="ref16">16</xref>
            ]
(RenewKube) supplied means to run Renew simulations containerized and
in an organized fashion with control from within the reference net simulation.
Throughout the development CI/CD by Gitlab9 has been used, as well as a
Scrum-based development approach.
          </p>
          <p>The remaining requirements can be evaluated in regards to the cloud native
Renew plugin on its own, but are more interesting when applied to the upcoming
redesign of the Distribute plugin, as it is the central part, that is in need of
resilience optimizations. However, for completeness, we consider them in the
context of the cloud native Renew plugin as well. As the endpoints are emitted
regardless of other components, failures (including the control panel) do not
a ect other components (requirement 8) and the simulation (requirement 9).
As there are no incorporated external services yet, requirement 10 is ful lled
trivially.</p>
          <p>Overall we could achieve a very solid detachment from local system access,
resulting in a reference net simulator, that is well suited to be run in cloud
environments. Upon completion of the remaining projects addressed throughout
the evaluation and paper, a thorough evaluation in a real cloud environment can
be conducted. The outlook section 7 will point to follow-up research topics in
this regard.
All shown limitations refer to our implementation. When using the simulation
controls, there is currently no feedback about the success or failure of the
operation. This is due to the concurrent nature of the simulator and the fact, that
HTTP server calls should not be implemented in a blocking fashion. Later this
can be solved using polling, webhooks, or websockets.</p>
        </sec>
        <sec id="sec-13-1-2">
          <title>9 https://about.gitlab.com/</title>
          <p>Detailed feedback about the simulation run is also not yet available in the
shape of a simulation feed. This also has implications on the Distribute plugin,
as distributed ring and recording of rings share similarities and should be
implemented together.</p>
          <p>
            There is also no cleanup mechanic implemented yet. Therefore very long
runs of plugins might require restarting of the containerized application to free
up space. This can be achieved using RenewKube [
            <xref ref-type="bibr" rid="ref16">16</xref>
            ], but is not yet built into
the application itself.
6.2
          </p>
        </sec>
      </sec>
    </sec>
    <sec id="sec-14">
      <title>Discussion</title>
      <p>Overall the contribution resembles groundwork for scalable applications, by
having a simulator, that can be run without machine-local interaction and without
need to access the underlying system. These aspects enable all kinds of
automation, by accessing respective endpoints. While introducing an increased
complexity, the interfaces are now much more streamlined and consumable by a variety
of tools.</p>
      <p>Within the prototype implementation new dependencies on third party
libraries have been introduced, to severely speed up the development process and
to avoid repeated work on solved problems at the risk of the possibility of
discontinuation of said dependencies. The method also introduces an overhead in
generating messages on the universal interface. However, as it also paves the way
for scabale applications, this e ect is expected to be compensated by the now
possible sizes of reference net applications.</p>
      <p>In the larger context of the multi-agent applications mentioned in the
motivation section, the cloud native plugin for Renew will serve as essential
platform extension to enable interoperability with an abstract platform management
(compound) system.
7</p>
      <sec id="sec-14-1">
        <title>Conclusion</title>
        <p>The conceptual integration of the cloud nativity paradigm into the Renew
simulator for reference nets was presented. While cloud nativity is a heavily used
term, especially in marketing departments, the here used interpretation is based
on the concepts of operability, observability, agility, and resilience. Necessary
changes to the architecture have been motivated and include the introduction of
an HTTP-based interface for monitoring and control features, the usage of agile
development, deployment via CI/CD, and usage of containerization. An
implementation of the aspects relevant to the here presented "cloud native plugin" for
Renew has been presented. It includes most of the observability and operability
aspects. Within the evaluation, it was shown, that the implementation met all
the relevant requirements. Overall we could provide a future proof improvement
over the dated Renew Remoting plugin and related solutions.</p>
      </sec>
    </sec>
    <sec id="sec-15">
      <title>Outlook</title>
      <p>
        Further work will include the implementation of the aspects concerning other
projects with relations to cloud native Renew, especially those regarding the
Distribute plugin and resilience aspects. Additionally, overall integration of all
related projects (Resilient Distribute, Petri net Sagas, RenewKube [
        <xref ref-type="bibr" rid="ref16">16</xref>
        ]) with
Cloud Native Renew is of interest. In addition, the next step is to deploy the
prototype in a common environment such as AWS or Azure to ensure that the
experimental approach works in a real cloud environment. Other open questions
address the safety of remote code execution operations regarding net and plugin
signatures.
      </p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          1.
          <string-name>
            <surname>Bernine</surname>
            ,
            <given-names>N.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Nacer</surname>
            ,
            <given-names>H.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>A</surname>
          </string-name>
          ssani,
          <string-name>
            <given-names>D.</given-names>
            ,
            <surname>Alla</surname>
          </string-name>
          , H.:
          <article-title>Towards a performance analysis of composite web services using petri nets</article-title>
          .
          <source>Int. J. Math. Oper. Res</source>
          .
          <volume>17</volume>
          (
          <issue>4</issue>
          ),
          <volume>467</volume>
          {
          <fpage>491</fpage>
          (
          <year>2020</year>
          ). https://doi.org/10.1504/IJMOR.
          <year>2020</year>
          .110847
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          2.
          <string-name>
            <surname>Bhandari</surname>
            ,
            <given-names>G.P.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Gupta</surname>
          </string-name>
          , R.:
          <article-title>Fault diagnosis in service-oriented computing through partially observed stochastic petri nets</article-title>
          .
          <source>Service Oriented Computing Applications</source>
          <volume>14</volume>
          (
          <issue>1</issue>
          ),
          <volume>35</volume>
          {
          <fpage>47</fpage>
          (
          <year>2020</year>
          ). https://doi.org/10.1007/s11761-019-00279-5
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          3.
          <string-name>
            <surname>Buchs</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Klikovits</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Linard</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Mencattini</surname>
            ,
            <given-names>R.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Racordon</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          :
          <article-title>A model checker collection for the model checking contest using docker and machine learning</article-title>
          . In: Khomenko,
          <string-name>
            <given-names>V.</given-names>
            ,
            <surname>Roux</surname>
          </string-name>
          ,
          <string-name>
            <surname>O.H</surname>
          </string-name>
          . (eds.)
          <article-title>Application and Theory of Petri Nets</article-title>
          and Concurrency - 39th International Conference,
          <source>PETRI NETS</source>
          <year>2018</year>
          , Bratislava, Slovakia, June 24-29,
          <year>2018</year>
          , Proceedings. LNCS, vol.
          <volume>10877</volume>
          , pp.
          <volume>385</volume>
          {
          <fpage>395</fpage>
          . Springer (
          <year>2018</year>
          ). https://doi.org/10.1007/978-3-
          <fpage>319</fpage>
          -91268-4 21
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          4.
          <string-name>
            <surname>Duvigneau</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          :
          <article-title>Konzeptionelle Modellierung von Plugin-Systemen mit Petrinetzen</article-title>
          . Dissertation, Universitat Hamburg, Department Informatik,
          <string-name>
            <surname>Vogt-Kolln Str</surname>
            . 30,
            <given-names>D</given-names>
          </string-name>
          <string-name>
            <surname>-</surname>
          </string-name>
          22527
          <string-name>
            <surname>Hamburg</surname>
          </string-name>
          (Oct
          <year>2009</year>
          ), https://ediss.sub.uni-hamburg.de/handle/ ediss/3023
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          5.
          <string-name>
            <surname>Entezari-Maleki</surname>
            ,
            <given-names>R.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Etesami</surname>
            ,
            <given-names>S.E.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Ghorbani</surname>
            ,
            <given-names>N.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Niaki</surname>
            ,
            <given-names>A.A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Sousa</surname>
            ,
            <given-names>L.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Movaghar</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          :
          <article-title>Modeling and evaluation of service composition in commercial multiclouds using timed colored petri nets</article-title>
          .
          <source>IEEE Trans. Syst. Man Cybern. Syst</source>
          .
          <volume>50</volume>
          (
          <issue>3</issue>
          ),
          <volume>947</volume>
          {
          <fpage>961</fpage>
          (
          <year>2020</year>
          ). https://doi.org/10.1109/TSMC.
          <year>2017</year>
          .2768586
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          6.
          <string-name>
            <surname>Fowler</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          :
          <article-title>Inversion of control containers and the dependency injection pattern</article-title>
          (january
          <year>2004</year>
          ), https://martinfowler.com/articles/injection.html
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          7.
          <string-name>
            <surname>Garcia-Molina</surname>
            ,
            <given-names>H.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Salem</surname>
            ,
            <given-names>K.</given-names>
          </string-name>
          : Sagas.
          <source>In: Proceedings of the 1987 ACM SIGMOD International Conference on Management of Data</source>
          . pp.
          <volume>249</volume>
          {
          <fpage>259</fpage>
          . SIGMOD '87,
          <string-name>
            <surname>ACM</surname>
          </string-name>
          , New York, NY, USA (
          <year>1987</year>
          ). https://doi.org/10.1145/38713.38742
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          8.
          <string-name>
            <surname>Garrison</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Nova</surname>
            ,
            <given-names>K.</given-names>
          </string-name>
          :
          <article-title>Cloud Native Infrastructure: Patterns for Scalable Infrastructure and Applications in a Dynamic Environment.</article-title>
          <string-name>
            <surname>O'Reilly Media</surname>
          </string-name>
          , Inc., 1st edn. (
          <year>2017</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          9.
          <string-name>
            <surname>Kilian</surname>
            ,
            <given-names>T.</given-names>
          </string-name>
          :
          <article-title>Entwicklung einer modularen JavaScript-Bibliothek zur Modellerstellung</article-title>
          . Bachelorarbeit, Universitat Hamburg, Fachbereich Informatik,
          <source>Vogt-Kolln Str</source>
          . 30,
          <string-name>
            <given-names>D</given-names>
            <surname>-</surname>
          </string-name>
          22527
          <string-name>
            <surname>Hamburg</surname>
          </string-name>
          (
          <year>2019</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          10.
          <string-name>
            <surname>Kummer</surname>
            ,
            <given-names>O.</given-names>
          </string-name>
          :
          <string-name>
            <surname>Referenznetze</surname>
          </string-name>
          . Logos Verlag, Berlin (
          <year>2002</year>
          ), http://www. logos-verlag.de/cgi-bin/engbuchmid?isbn=0035&amp;lng=eng&amp;id=
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          11.
          <string-name>
            <surname>Kummer</surname>
            ,
            <given-names>O.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Wienberg</surname>
            ,
            <given-names>F.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Duvigneau</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          , Kohler,
          <string-name>
            <given-names>M.</given-names>
            ,
            <surname>Moldt</surname>
          </string-name>
          ,
          <string-name>
            <surname>D.</surname>
          </string-name>
          , Rolke, H.:
          <article-title>Renew { the Reference Net Workshop</article-title>
          . In: Veerbeek,
          <string-name>
            <surname>E</surname>
          </string-name>
          . (ed.)
          <source>Tool Demonstrations. 24th International Conference on Application and Theory of Petri Nets (ATPN</source>
          <year>2003</year>
          ).
          <source>International Conference on Business Process Management (BPM</source>
          <year>2003</year>
          ). pp.
          <volume>99</volume>
          {
          <fpage>102</fpage>
          . Department of Technology Management, Technische Universiteit Eindhoven,
          <source>Beta Research School for Operations Management and Logistics (Jun</source>
          <year>2003</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          12.
          <string-name>
            <surname>Lacheheub</surname>
            ,
            <given-names>M.N.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Hameurlain</surname>
            ,
            <given-names>N.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Maamri</surname>
          </string-name>
          , R.:
          <article-title>Resources consumption analysis of business process services in cloud computing using petri net</article-title>
          .
          <source>J. King Saud Univ. Comput. Inf. Sci</source>
          .
          <volume>32</volume>
          (
          <issue>4</issue>
          ),
          <volume>408</volume>
          {
          <fpage>418</fpage>
          (
          <year>2020</year>
          ). https://doi.org/10.1016/j.jksuci.
          <year>2019</year>
          .
          <volume>08</volume>
          .005
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          13.
          <string-name>
            <surname>Moldt</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          , Rowekamp,
          <string-name>
            <given-names>J.H.</given-names>
            ,
            <surname>Simon</surname>
          </string-name>
          ,
          <string-name>
            <surname>M.:</surname>
          </string-name>
          <article-title>A simple prototype of distributed execution of reference nets based on virtual machines</article-title>
          . In: Bergenthum,
          <string-name>
            <given-names>R.</given-names>
            ,
            <surname>Kindler</surname>
          </string-name>
          , E. (eds.)
          <source>Algorithms and Tools for Petri Nets Proceedings of the Workshop AWPN</source>
          <year>2017</year>
          ,
          <article-title>Kgs</article-title>
          . Lyngby,
          <source>Denmark October 19-20</source>
          ,
          <year>2017</year>
          . pp.
          <volume>51</volume>
          {
          <fpage>57</fpage>
          .
          <source>DTU Compute Technical Report</source>
          <year>2017</year>
          -
          <volume>06</volume>
          (
          <year>2017</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          14.
          <string-name>
            <surname>Richardson</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          :
          <article-title>Microservices Patterns: With Examples in Java</article-title>
          . Manning
          <string-name>
            <surname>Publications</surname>
          </string-name>
          (
          <year>2019</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref15">
        <mixed-citation>
          15. Rowekamp, J.H.:
          <article-title>Investigating the java spring framework to simulate reference nets with Renew</article-title>
          . pp.
          <volume>41</volume>
          {
          <fpage>46</fpage>
          . No.
          <year>2018</year>
          -02 in Reports / Technische Berichte der Fakultat fur Angewandte Informatik der Universitat
          <string-name>
            <surname>Augsburg</surname>
          </string-name>
          (
          <year>2018</year>
          ), https:// opus.bibliothek.uni-augsburg.de/opus4/41861
        </mixed-citation>
      </ref>
      <ref id="ref16">
        <mixed-citation>
          16. Rowekamp,
          <string-name>
            <given-names>J.H.</given-names>
            ,
            <surname>Moldt</surname>
          </string-name>
          ,
          <string-name>
            <surname>D.:</surname>
          </string-name>
          <article-title>RenewKube: Reference net simulation scaling with Renew and Kubernetes</article-title>
          . In: Donatelli,
          <string-name>
            <given-names>S.</given-names>
            ,
            <surname>Haar</surname>
          </string-name>
          , S. (eds.)
          <article-title>Application and Theory of Petri Nets</article-title>
          and Concurrency - 40th International Conference,
          <source>PETRI NETS</source>
          <year>2019</year>
          , Aachen, Germany, June 23-28,
          <year>2019</year>
          ,
          <source>Proceedings. Lecture Notes in Computer Science</source>
          , vol.
          <volume>11522</volume>
          , pp.
          <volume>69</volume>
          {
          <fpage>79</fpage>
          . Springer (
          <year>2019</year>
          ). https://doi.org/10.1007/978-3-
          <fpage>030</fpage>
          - 21571-2 4, https://doi.org/10.1007/978-3-
          <fpage>030</fpage>
          -21571-2
        </mixed-citation>
      </ref>
      <ref id="ref17">
        <mixed-citation>
          17. Rowekamp,
          <string-name>
            <given-names>J.H.</given-names>
            ,
            <surname>Moldt</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            ,
            <surname>Feldmann</surname>
          </string-name>
          ,
          <string-name>
            <surname>M.</surname>
          </string-name>
          :
          <article-title>Investigation of containerizing distributed Petri net simulations</article-title>
          . In: Moldt,
          <string-name>
            <given-names>D.</given-names>
            ,
            <surname>Kindler</surname>
          </string-name>
          ,
          <string-name>
            <surname>E.</surname>
          </string-name>
          , Rolke, H. (eds.) Petri Nets and
          <string-name>
            <given-names>Software</given-names>
            <surname>Engineering</surname>
          </string-name>
          . International Workshop, PNSE'
          <fpage>18</fpage>
          , Bratislava, Slovakia, June 25-26,
          <year>2018</year>
          .
          <source>Proceedings. CEUR Workshop Proceedings</source>
          , vol.
          <volume>2138</volume>
          , pp.
          <volume>133</volume>
          {
          <fpage>142</fpage>
          .
          <string-name>
            <surname>CEUR-WS.org</surname>
          </string-name>
          (
          <year>2018</year>
          ), http://ceur-ws.
          <source>org/</source>
          Vol-
          <volume>2138</volume>
          /
        </mixed-citation>
      </ref>
      <ref id="ref18">
        <mixed-citation>
          18.
          <string-name>
            <surname>Simon</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Moldt</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          :
          <article-title>Extending Renew's algorithms for distributed simulation</article-title>
          . In: Cabac,
          <string-name>
            <given-names>L.</given-names>
            ,
            <surname>Kristensen</surname>
          </string-name>
          ,
          <string-name>
            <surname>L.M.</surname>
          </string-name>
          ,
          <string-name>
            <surname>R</surname>
          </string-name>
          olke, H. (eds.) Petri Nets and
          <string-name>
            <given-names>Software</given-names>
            <surname>Engineering</surname>
          </string-name>
          . International Workshop, PNSE'16,
          <string-name>
            <surname>Torun</surname>
          </string-name>
          , Poland, June 20-21,
          <year>2016</year>
          .
          <source>Proceedings. CEUR Workshop Proceedings</source>
          , vol.
          <volume>1591</volume>
          , pp.
          <volume>173</volume>
          {
          <fpage>192</fpage>
          .
          <string-name>
            <surname>CEUR-WS.org</surname>
          </string-name>
          (
          <year>2016</year>
          ), http://CEUR-WS.org/Vol-1591
        </mixed-citation>
      </ref>
      <ref id="ref19">
        <mixed-citation>
          19.
          <string-name>
            <surname>Valk</surname>
          </string-name>
          , R.:
          <article-title>Petri nets as token objects - an introduction to elementary object nets</article-title>
          . In: Desel,
          <string-name>
            <given-names>J.</given-names>
            ,
            <surname>Silva</surname>
          </string-name>
          , M. (eds.) 19th
          <source>International Conference on Application and Theory of Petri nets</source>
          , Lisbon, Portugal. pp.
          <volume>1</volume>
          {
          <fpage>25</fpage>
          . No. 1420
          <source>in Lecture Notes in Computer Science</source>
          , Springer-Verlag, Berlin, Heidelberg, New York (
          <year>1998</year>
          ). https://doi.org/10.1007/3-540
          <source>-69108-1 1</source>
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>