Reference nets combine the power of an object-oriented programming language with the formalism of Petri nets. Using them makes it possible to model complex concurrent systems easier. By doing so with a tool like Renew, we achieve a directly executable (local) simulation of the system. In recent publications, ideas have been presented to transpose these simulations into one distributed simulation. However, being a genuinely concurrent system, the problem arises, that some nodes are less utilized than others slowing down the simulation in its entirety. In this paper, we define requirements to a distributed simulation and evaluate how specific technical approaches harmonize with them. We present a first - reference net based - prototype for an approach using containerization and Docker for the execution of a single net simulation truly distributed on several (virtual) machines. As an example of a workload with differing complexity, the prototype calculates parts of the Mandelbrot set in a distributed manner. A significant advantage of the approach is that inherent complexity of a problem may be unknown without restraining the method.
Reference nets are a high-level Petri net formalism and provide a strong
objectoriented foundation to model complex systems. Because reference nets follow the
principal of nets within nets, use net instances (similar to object orientation)
and are organized hierarchically, communication between nets imposes a certain
overhead. Synchronous channels are a practical solution in local cases. However
to find a binding unification is required, which gets complicated in the context
of synchronous channels in the distributed case. A solution using very simple
synchronous channels has been proposed [
When simulating reference nets, it is desirable to not only simulate a distributed system but to distribute the simulation altogether. Having a simulation infrastructure that copes with different complications inherent to a distributed system, gives the possibility to achieve this goal. Using a distributed net simulation has several advantages, for example, it benefits all parts of the software engineering life cycle. It can be used to model prototypes and mock-ups during requirements engineering, implement the actual system during implementation or help with verification during maintenance. 2 2.1
Some Petri Net basics of reference nets and their toolset are explained, as they form the basis for our conceptual and technical proposal.
sidered as monolithic structures. High-level Petri nets introduced hierarchies and allowed splitting of nets into subcomponents of the whole structure. Binding of subnets with the embedding nets has been discussed intensively.
Reference nets [
In Figure 1 the left net is the System net called SystemNet and other net is the Object net called subnet. An instance of the SystemNet net template is instantiated, fires the first transition and generates a token that contains a string "Demo", an integer 42 and the reference to the generated Object net instance of a subnet net template called subnet. Binding of the two transitions via the channel ch is first collecting the possible bindings in the SystemNet, binding it to the parameters of the channel (str will be bound to "Demo", x to 42 and subnet to the net instance reference in this example). Subsequently, the binding of the nonbound parameters attached to the transition in the subnet is calculated (a possible binding in the example would be localY to 42, givenStr to the same value as str and by that "Demo", and givenInt to the value of x, which is also 42). Then the firing of the transitions in both nets occurs concurrently. Resulting in the marking of 1764 (the value, that will be bound to prod) in the right most place in the SystemNet and the tuple ["Demo,42"] in the right most place in the Object net. Please note the asymmetric call: the caller (SystemNet) has to have the reference of the callee (subnet) to invoke the channel ch with subnet:ch(str,x,prod).
Renew For reference nets, the tool Renew has been developed. While it can be used for many other formalisms reference nets and its channels are the primary formalism. It allows the modeling and the execution of reference nets in a genuinely concurrent mode.
In [
Other formalisms than the reference nets of Renew allow slightly modified channel inscriptions. However, they provide only synchronous channels for local calls of other nets. Communication with other machines is only realized via other frameworks like RMI, ActiveMQ, and others. 2.2
Virtual machines are designed to run applications on a particular system. A
hypervisor is located on top of the infrastructure and creates and runs virtual
machines [
Divergent to virtual machines, containers are located on top of the
infrastructure and its host OS. The containers share the OS Kernel and may share the
binaries and libraries [
Container management is responsible for all actions concerning containers [
Autoscaling Multiple containers of the same image can be used to split a load
across multiple containers. Autoscaling is responsible for adjusting the number
of containers to meet the demand [
For further evaluation, it is desirable to outline our requirements for a distributed simulation. To run a distributed simulation several (technical) aspects need to be considered. The three main categories of interest are: The net formalism, the infrastructure and finally the engine used to run the simulation. We assume that all these are possibly alterable. As a distributed simulation itself is also a distributed system, well-known requirements for distributed systems apply as well. Therefore, resources are a crucial term in all three categories. A distributed simulation should keep the resource demand as low as possible. Also, derived from distributed systems, security concerns are to be considered. This is mainly an infrastructural topic, but to a degree applies to the engine itself and the net formalism as well. In our context security is a generalized term on how good components of the net are protected from unauthorized access, regardless of it originating from within or without the simulation itself. Finally, a net formalism related requirement is splitability and distributability of net parts. While in an ideal world every concurrent transition firing could run distributed as well, this is very hard to achieve due to synchronization constraints. However, achieving a good distributability is one major goal in designing a distributed simulation environment.
Partitioning of the simulation is crucial for a distributed simulation. Out of the net formalisms we choose reference nets, because of their net instance capabilities. Similar to object-oriented programming nets are like classes, of which net instances can be created. Net instances themselves can be treated as a token in a place in another net instance. Being able to create net instances on demand results in significantly increased flexibility. However, there is a drawback: The resource demand of the simulation may increase during running time. This also means that the simulation environment might not be known in its entirety when starting the simulation. To be able to dynamically provide more computation power, in case the simulation is already fully utilizing the underlying system, we investigate three kinds of infrastructure and their capabilities of extending the simulation on demand. These three main approaches are bare metal machines, virtual machines, and containers. 3.1
The classic approach is to start one part of the simulation directly on a physical machine. This method has the least overhead but is also somewhat static. Adding an additional unit requires a fully installed machine, that is connected to the network. The simulation also needs to wait until the new machine is booted before it can be used. There is also a significant installation overhead to be considered, as all components need to be installed before the first run. 3.2
The next logical step is to migrate the distributed simulation into virtual
machines as we have proposed earlier [
The main shortcomings of the virtual machine approach are size and boot time. The usage of containers can elegantly address both. As mentioned earlier Docker as a container virtualization technology comes with integrated security features despite sharing resources on a machine. Using containers finally allows for portable, isolated simulation parts, that can be added to the simulation promptly. 4
Per assumption (see section 2.4) net formalism, infrastructure and engine may be altered. Using Renew (with Distribute plugin) we can apply changes to the base formalism of reference nets and also extend the engine using plugins. Distributed synchronous channels are an example of an element diverging from the base reference net formalism. As this work examines the use of containers a possibly changing underlying infrastructure is of primary interest. We present a brief prototype to show the general extendability of a simulation using Renew with Distribute plugin. Extendability is a necessary precondition to be able to adapt to changing infrastructure. Renew is technically a Java extension. However, it provides true concurrency, non-determinism and conflict management (fairness). Also note, that the Distribute plugin is based on Java RMI, but completely abstracts this fact from the modeler. 4.1
A central work distributor will be used to offer work packages. Other parts of the simulation can synchronize with it to fetch packages and deliver done work. These other parts are realized within prepared (container-)images, so by merely launching such an image into a container, the simulation gets extended. To realize a proper utilization of the system, partitioning the workload is desirable in such a manner every simulation node is utilized. Note, that we use a centralized approach (instead of, e.g., work stealing) to avoid the necessity for each (maybe newly initialized) worker to know each other currently active worker. 4.2
With the proposed prototype we mainly want to evaluate the hypothesis, whether it is possible to dynamically extend a running reference net simulation during runtime using containerization technology. As a secondary goal, a high system load is desirable to avoid inefficient utilization of resources. 4.3
The concept is realized using Docker as container technology, Renew 2.5 as
simulator using the ‘Distribute’ plugin [
There are several possible application scenarios, like modeling of a complex
system, multi agent applications, software verification and more. However, all of
these are rather complex and since this prototypes intended use is to evaluate,
whether a simulation is dynamically extendable with containers, a more
straightforward task was chosen. The computation and visualization of the Mandelbrot
set [
To demonstrate the difference between the dynamic distribution of work
packages and the static distribution onto a fixed number of instances, the image
is computed twice in two alternative ways: First, the image is split into four equal
parts. Four workers are launched and each worker processes its equal sized part
of the problem. Second, the image is split into 16 parts. These are dynamically
assigned to newly created containerized simulation parts (workers). By doing so
the second alternative automatically adds computation agents to the simulation,
which is a very basic form of autoscaling (see section subsec:containerMgmt)
without dynamic load adaption. Both alternatives are calculated with the same
nets, only the configuration is different to implement these two approaches. The
blueprint for a worker net can be found in figure 3 and the distributor net in
figure 5. A launcher net starts new net parts (docker containers). The launcher
is shown in figure 4. Please note that the transition inscription for the channel
differs from the channel explained in Figure 1: d!passWorkload() -> wp. The
syntax is part of the distributed reference net formalism and required for the
binding search, but are too complicated to elaborate on here. As for intuition
imagine -> as output parameter of a firing, <- as input parameter and ! as
synchronization with a non-local net instance. For further details see [
Different approaches using bare metal machines, virtual machines and containerization have been discussed. Finally, a simple prototype using the containerization technology has been shown.
Future work can include but is not limited to fully fledged reference net based autoscaling, net partitionability considerations, and evaluation of more complex net tasks.