In this contribution we study organisation-centred multi-agent systems (Org-MAS), i.e., MAS where the agents are embedded into an explicitly modelled organisational structure with positions, roles, hierarchies etc. Whenever an Org-MAS adapts this happens at two levels: At the micro-level the agents learn and at the macro-level the organisation adapts its structure. These two learning processes are intertwined and this co-learning is known as the micro-macro-link in sociology. In previous work, we have developed an execution engine for Org-MAS, called Sonar, specified using Nets-within-Nets, i.e., Petri nets which contain other Petri nets as tokens. The learning part of the engine defines the micro-macro-link of two interconnected MAPE-like (monitor, analyse, plan, execute) learning processes. The Sonar-engine uses a digital twin of itself, called the Sonar-MAPE-Loop@run.time, during the planning of MAPE to predict benefits and costs of applicable adaptation steps to allow for a goal-directed Adaptation. In this paper we study the behavioural impact of the meta-parameters that are used by the digital twin for the prediction. Additionally, we explore the aspect that meta-parameters of the twin correspond to deployment variants of the organisation. Here, we demonstrate how to use our twin model to find a 'good' organisation design using a sweep through the parameter space.
In general, we are interested in the design and analysis of self-adaptive systems [
The central concepts of MAS (like autonomy, rationality, and cooperation) are complemented
by organisational aspects (like roles, norms, positions, protocols, etc.). This combination is
known as organisation-centered MAS (Org-MAS) [
The Sonar-Execution Engine An MAS specified with Sonar is executed by an execution
engine [
This execution engine supports adaptation via a monitor-analyse-plan-execute loop, MAPE for
short [
For the adaptation of the Sonar organisation we have several means: We could adjust the organisational structure by adding new protocols or by structural modifications of the organisation net, i.e., adding delegation options, etc. A second approach is to modify the state of the organisation that controls team formation processes: For each task to be performed by an MAS an organisation model defines a whole set of possible teams. Which team is chosen for a concrete task at run-time depends on the organisational state. In this paper we concentrate on the second, simpler kind of modification of the organisational state; the structural adaptation is subject to ongoing work.
Using a Digital Twin during the Planning Step In our Sonar-MAPE-Loop we monitor
the execution data and performance indicators at run-time to detect problematic spots, like
bottle-necks, and we analyse, which parts of the organisation (the network, the interaction
patterns, etc.) are interesting candidates for an adaptation [
In general, we cannot predict the change of indicator values for the Org-MAS candidate
by a purely static analysis due to the complexity. Instead, we simulate a digital twin of the
Org-MAS candidate, called the Sonar-MAPE-Loop@run.time, during the planning, i.e., we
follow a models@run.time approach [
The use of nets-as-tokens (another name for nets-within-nets [
The meta-parameters correspond to a concrete deployment of our MAS; e.g., a low value corresponds to a deployment where organisational constraints are, more or less, only recommendations for the member agents, while a high value of corresponds to an enforcement of organisational constraints. These diferent deployments difer in their complexity, e.g., enforcement of organisational constraints has to be implemented by more complex techniques. Therefore, an analysis of the efect of the meta-parameters for a given organization model helps the designer to identify those deployment configurations that have a low complexity and enable a good performance at the same time.
Research Agenda On the one hand, an organisational design has to be restrictive in order to help agents to make good decisions (short-term reward); on the other hand, it must give the agents room for experiments to adapt to changes flexibly (long-term reward). Therefore, a central aspect of a ‘good’ Org-MAS design is the balance between the organisational constraints and the individual decisions within a long-term co-evolutionary adaptation process, i.e., the micro-macro dynamics. When considering the long-term behaviour we also favour ‘robust’ organisations, i.e., MAS where the behaviour does not depend critically on the given configuration. Instead we want that a variation of parameters has either little impact or the system will stabilise soon – maybe ‘somewhere’ else.
Additionally, quality is multi-dimensional and therefore a change in the organisation might
be beneficial for one dimension, but worse for another. The central discovery of computational
organisation theory [
Related Work The most prominent paper to adaptive systems is given in the context of self-*
properties (self-healing, self-organising, etc.) [
The theoretical foundations of adaptivity are formulated for Complex Adaptive Systems
(CAS) [
The central approach to adaptivity in software engineering is known as the MAPE-K
approach, i.e., MAPE with K (knowledge), where the knowledge has much in common with
the organisational concepts [
The work presented here also has some connections to our research on adaptation state spaces
[
Prominent examples of Org-MAS from the design and implementation perspective are
AGR [
In the following we give a short introduction into the Sonar formalism, which is used for
analysis, as well as for implementing an Org-MAS. The Sonar organisational model [
In MAS, organisations are used to describe super-individual control concepts to complement
the purely local perspective of agents and to overcome the so-called price of anarchy. These
concepts are usually introduced in MAS organisations [
The interaction protocols are defined by a distributed workflow net , which is a multi-party
version of the well-known workflow nets (WF nets; WFN) [
In Sonar, the organisation is a Petri net = (, , ), where each place is of the form = task[], which describes a task for the agent to establish the behaviour that is described by the subnet []. Our standard working example of an organisation net is shown in Figure 1. It will be explained in more detail in Section 3.
The tasks are either generated in the environment or are sub-tasks, generated by the organisation itself. The places with empty pre-set are those tasks that the organisation is responsible for, i.e., tasks that are generated externally: 0(Org ) := ∘ := {0 ∈ | ∙ 0 = ∅}.
Each task is handled by the transition of the organisation net, which are called team-operators. We have four types of operators: 1. Delegate: The task to implement [] is delegated from agent to . Only the delegation operation delegates the ownership of a task. 2. Split: The task to implement = {1, . . . , } in is split into sub-tasks to implement {} in . 3. Refinement: Here, [] is replaced by ′[1, . . . , ], which has to be a behaviour equivalent refinement, i.e. they are bisimilar. 4. Assign execution: The WFN [] is assigned to an agent for execution.
Each team-operator also imposes a constraint () onto the execution of WFN . Each subset of these operators induces a Sonar-Organisation Net = (, , ) where = ∙ ∪ ∙ . The mapping : → returns the owner of a task: (task[]) := . The mapping is extended to transitions by defining () := () for the unique place in the preset of .
A Sonar-Organisation Org = (, , ℛ, ) is given by the organisation net , the set of positions , which is the set of agents owning the operators: = ( ), the role set ℛ, and the set of WFN . The agents in are called organisation position agents (OPA).
The Sonar-engine defines an execution loop to carry out the organisational teamwork. The execution loop is carried out by a MAS, called the OPA-OMA-Network.
We use Sonar models to generate MAS architectures. This is done by parameterising a general Org-MAS engine with a concrete Sonar model. The Sonar-engine instantiates each position ∈ of the model by an agent: the organisation position agent (OPA). The network of OPAs as given in the model Org = (, , ℛ, ) defines the so-called formal organisation. An OPA roughly represents the type of agents expected at the given position. The implementation of this position is done by coupling another agent, the organisational member agent (OMA), with the OPA. The OMA is not part of the formal model and is implemented separately. Note, that there is no need for additional concepts to integrate organisations into our MAS, as we specify the organisation as a network of OPAs and this network is just a part of the whole system. The OPA-OMA network represents the Sonar model at run-time.
Formally, teams are partial-order runs [
For the organisation in Figure 1 the highlighted nodes define such a partial-order run, i.e., a team , for the initial task = taskPC1 [Prod,Cons]. (Recall that this describes a task for the organisation agent 1 that is handled by the interaction of the two roles Prod and Cons as specified in the WFN PC .) The three transitions labelled with a hammer at the agent 21, 2 and 4 show the assignment operators, i.e., these agents really implement the roles in the team-WFN . The team shown here will execute a team-WFN (), that is generated as the dynamic composition of the services provided by the three final transitions: () = (PC 3[Prod 1] ‖ PC 3[Prod 2] ‖ PC [Cons])
The transition labelled with light bulbs denote ‘inner’ agents. They do not participate directly in the team interaction, but, since the constraints () of inner agents have to be fulfilled in the execution of (), too, these agents still fulfill a coordinating purpose.
Simplifying a little, a central aspect of the Sonar-teamwork is to dynamically select the team-WFN () at run-time from the set of all WFN that are allowed by the organisation during a cooperative process among the agents; the organisational structure guarantees that () is a refinement of the initial WFN [] for the considered initial task = task[].
The Sonar-engine defines a general execution loop where a task triggers the formation of a
team of agents. The Sonar-MAPE-Loop is presented in [
We use nets-within-nets [
1This run-time engine is a revised version of a former approach presented in [
We will give a rather short description of the net, details are given in [
Note, that there is no need for additional features in our model to allow for transformations as the Hornet-model of the Sonar-MAPE-loop in Figure 2 contains two kinds of protocols (i.e. workflows): the normal first-order workflows, which modify the agents’ environment (as discussed here); but, the model also contains second-order workflows (not discussed here), which modify the first-order workflows (using a synchronisation with the transition modify wfn in Fig. 2) and/or the Sonar-model itself (synchronising with the transition modify team operators). In principle, this hierarchy is unbounded, as we allow for third-order workflows to modify second-order workflows and so on.
For the second-order protocols, which are used for adaptation, we have to estimate the efect on the system (cost-benefit reasoning) . This is done in the planning phase of the MAPE-loop using a digital twin.
During the adaptation of the organisation (as performed by the transition adapt configuration in Figure 2) we have to predict the impacts of modifying the Sonar configuration (the organisation state) to ′. This is complicated by the fact that changes may lead to a diferent micro-macro dynamics as well. The transition uses a digital twin of the whole Sonar-MAPELoop (not shown in Fig. 2) to predict the efect of adaptations.
Since we use the digital twin for predicting the outcome of several adaptation candidates the simulation of the twin model has to be much faster than the system itself. Therefore, the twin has to be an abstraction. The digital twin also has to include the external elements, i.e., the environment and the member agents. Both are given as a stochastic model assigning probabilities to alternatives (instead of a complex decision logic). The interplay of organisation, member agents, and environment is modelled by our meta-parameters org , tra , and , explained in the following subsections.
To predict the resulting micro-macro dynamics we simulate a digital twin of the Sonar-MAPELoop. The digital twin is a simplified version of the original Sonar-MAPE-Loop net: We model all internal agents’ logic by giving choices a probability. These choices are present in the team
ctra
η corg
transparency , and learning step size WFN (as xor-choices) and in the organisation net (as diferent branches for the team formation) as well.
Each organisational position agent (the OPA) is coupled with its member agents (the OMA) via the meta-parameter : As the Sonar specification formulates constraints in a probabilistic way (i.e. defining a probability OPA for choices of the OPA) and the OMA is modelled as probabilistic actor (i.e. defining a probability OMA), too, the overall behaviour is simply a superposition of both: Here, the meta-parameter ∈ [0; 1] denotes the organisational impact, i.e., to which extend members are obliged to fulfill organisational requirements. Here, we have two borderline cases: coherence. • = 0: the organisation has no impact at all and only the choices of the member agents matter. This usually describes quite flexible systems, but often with only limited • = 1: the organisational constraints are strong obligations. This leads to maximal predictability, but the system is rather inflexible, because agents cannot deviate.
The agents learn a decision probability for each choice. In our model, agents learn by following an environmental feedback signal , which they try minimise by going steps of size ≥ (also known as the learning rate) into the opposite direction of the gradient ′. The agents 0 also react on the organisational constraint () at time by incorporating · () into their value (). The weight ∈ [0; 1] describes the transparency of the organisation about its preferences (). A high value describes an organisation that is quite clear in communicating the constraints and therefore the agents will incorporate them faster. The combination of theses two aspects leads to the learning formula: ( + 1) = (1 − ) · ︁( () − · ′(︀ ())︀ ︁) impact , the organisational transparency , and the learning step size . (1) (2)
We are interested in diferent aspects of the Org-MAS behaviour w.r.t. the meta-parameters. • Quality Impact: For which configurations of the meta-parameters is the ratio of ‘resulting value’ to ‘optimal value’ greater than a given threshold?
This is the most obvious question as these configurations define the candidate set. • Interchangability: For which configurations do we obtain the same ratio of ‘resulting’ to ‘optimal’ behaviour? This is interesting because for these configurations one can trade stronger impact of organisational constraints for a lesser individual learning rates or a lower transparency – or vice versa. For many scenarios the designer is faced with given learning capabilities of the member agents. For interchangeable configurations we can adjust the organisational impact accordingly.
Extension: Which interchangeable configurations have a kind of fixed ‘exchange-rate’ of learning/transparency and organisational constraints? The designer can switch between those configurations. So, the designer can balance the costs for enforcing organisational constraints with the learning costs of members according to the concrete deployment environment. • Indispensability: For which configurations is it not possible to compensate a lower organisational impact by a higher learning rate? In other words: Are there configurations where a lower organisational impact reduces the ratio, more or less, independently from the learning rate? These configurations are the contrary of the interchangeable ones. They indicate that organisational coordination is essential and the designer should not lower the impact value. • Visibility: Are there configurations such that a lower organisational impact may be compensated only if the agents have a high learning activity and, at the same time, the organisation has a strong transparency? In this case the designer has to make sure that member agents are not only reacting to the environment but also incorporate organizational constraints into their decisions. • Cumbersomeness: Are there configurations where a higher organisational impact and/or incorporation activity is contra-productive (i.e., leads to a worse ratio)? In this case the designer has to moderate the impact of the organization and also the tendency of incorporating them.
In the following we study some examples to illustrate these properties.
In the following we explain our properties giving some examples where we vary the
metaparameters. The following scenarios are all executed as instances within our
Sonar-MAPELoop. The whole system uses Renew [
Two agents produce one of two possible goods: or . Each agent has a preference on how much he will produce of each good. The interaction protocol defining the production process contains two roles: 1 and 2 for the two producers.
The organisation net is a quite simple one. There is exactly one task, the production of resources and , and the organisation model induces exactly two teams for handling this task.
We have three OPAs: 0, 1 and 2. Here, the main decision, which of the teams is chosen, is made by 0. For both teams the agent 1 is responsible for role 1 and 2 for 2. The teams difer in the organisational constraints imposed on 1 and 2.
For S0 we assume that the environment exhibits a quite simple (and deterministic) behaviour: it oscillates each = 500 production steps (which equals the number of executed loops in the WFN) between two diferent states (i.e., quite slowly). The OPA 0 adapts to this external change by switching between the two teams. Essentially this means that the production probability constraint of good is = 0.6 for the first state and = 0.1 for the second.
We have two OMAs (i.e., member agents). They start with an initial preference, too. The initial preference for good is 1() = 0.7 for the first agent and 2() = 0.5 for the second at time = 0. The member agents adapt at the end of each execution of the WFN. In this basic scenario an agent adapts its behaviour towards the organisational constraints: ( + 1) = (1 − ) · () + · (), = 1, 2 (3)
Therefore, S0 is a special case of (2) from Section 4.2 with no learning from environmental feedback: = 0.
The second scenario is similar to S0. We still produce two kind of resources, and the desired production changes each = 500 steps. We still oscillate between two states. The first state still describes an environment that favors a production outcome where resource a has a frequency of = 0.6. But now the second states does no longer favor a single choice – now it prefers two alternative outcomes: Either there is a quite low frequency − or a quite high frequency +. These two values are the maximal values of our hat-like reward function are shown in Figure 4 (a).
The existence of two distinguishable options make the situation a kind of a coordination game (Therefore, our scenario bears a faint resemblance to the battle-of-sexes in game theory.): Both agents should make the same choice, since if they take opposite choices the resulting frequency will be around 50%, which is the worst outcome. We assume that a high learning rate is not able to compensate for a low organisational impact as this is also true for the battle-of-sexes game.
Our intention is the following: Since the member agents adapt independently, there is the risk that they will adapt their selection preference in an uncoordinated way, i.e., in diferent directions in our case. This risk is known as the price-of-anarchy in game theory. Organisations are supposed to compensate the unwanted efects of uncoordinated adaptation steps. 0,25 -1,25 -1 -0,75 -0,5
0,75 Michael Köhler-Bußmeier et al. CEUR Workshop Proceedings
We will start -o0,5ur OMA for role 1 with a rather high preference 1 = 0.7 for resource and the OMA for 2 with low one of 2 = 0.4. In this case it is quite likely that the OMA for 1 will adapt towards th-0,7e5 high optimum +, while the OMA for 2 will adapt in the opposite direction towards the low optimum − .
For simplicity we assume that the two alternatives − and + are equally attractive, i.e., we have the alternative minima ± = 0 ± . Moreover, we like the function to be symmetric. Here, we choose a reward function of the following form to model the hat-like shape of Fig. 4 (a): () = − ( − 0)2 · (( − 0)2 − 22) Adaptation (maximising the reward ) is equivalent to searching for a minimum of the loss function () := − ().
For S1 the member agents adapt only via a gradient descent (of step size ≥ 0): ( + 1) = () − · ′(()), = 1, 2 (4)
Therefore, S1 is a special case of the learning rule (2), where we ignore the incorporation of organisational constraints: = 0.
In this scenario will still toggle between two states each = 500 time steps. For uniformity, we model also the first state (where = 0.6 is desirable) by a function of the shape above, only with a very narrow distance between the two minima: We choose 0 = 0.6 and = 0.1, which leads to minima at − = 0.5 and + = 0.7. For the second state, we have 0 = 0.5 and = 0.3 resulting in − = 0.2 and + = 0.8. The loss functions for the two states are sketched in Figure 4 (b).
For S1 we assumed that agents only follow the feedback from the environment. Scenario S2 will also incorporate the feedback signal from the organisation and we use our general update rule (2), i.e., we combine the learning functions (3) and (4) from S0 and S1. Therefore, S2 now relates all three meta-parameters at the same time.
Scenario S3 modifies S2 slightly, such that the organisation still performs its role as it reduces complexity by breaking the symmetry towards the lower production frequency. But, we assume that we are now at the point where the environment has changed while the organisation is still adapted to a former situation not valid anymore.
In the previous scenarios the organisational decisions were always ‘correct’. However, even if the organisation has managed to configure itself in such a perfect way, sooner or later the rewards will change due to a dynamic environment. Therefore, we have the risk that the organisation is not only badly adjusted; even worse, it will hamper the member agents, which are usually much quicker in adapting to a change in the environment. In such a situation the organisation rules are not the solution to overcome the price-of-anarchy; instead they are a millstone to adaptation. We assume, that a good compromise between these two extremes will position the value of somewhere around ≈ 0.5, but, of course, the concrete value will depend much on the frequency and the strength of environmental changes. The parameter sweep in Sect. 7 will quantify this compromise region.
In the following we evaluate the simulation results with respect to their accordance to our hypotheses, which are about the general relationship between the meta-parameters. We evaluate the digital twin with a finite horizon ℎ of oscillations. Here, we use ℎ = 8 (which turned out to be computational feasible and big enough to observe the general trend), which leads to a number of ℎ · = 4000 micro-adaptations of the OMAs.
Note, that for the scenarios S0 - S3 the desired behavior coincides with the organisational constraint, while for S5 the organization is slightly wrong. Thus, the designer can identify good values for the meta-parameters by the degree of how closely the systems’ production follows the organisation line.
Sheet2
Sheet2 1 2 3 4 5 6 7 8 Cmm = 0.5 OMA-adaptC = 0.001 p1 p2 p_Org res A
res B OMA-adaptC = 0.001 OMA-adaptC = 0.002 1 2 3 4 5 6 7 8 Cmm = 0.5 OMA-adaptC = 0.002 70 p1 64 2p245
p_Org corg = 0.0 55 35 15 corg = 0.25 5355
15 meta-parameters, especially the organisational impact and the learning rate of member agents.
Figure 5 shows the adaptation of the member agents. It shows the selection probability , = 1, 2 for resource over time for = 0.001 (left) and = 0.002 (middle) in comparison to the change of the organisational selection probability . One can clearly observe that with the higher adaptation rate = 0.002 the member agents follow the organisational selection constraints more closely.
Figure 6 shows the impact of the micro/macro mixing-parameter . The diagrams show the frequency of resource taken for increasing values of . The simulation uses = 1.0, 0.75, 0.5, 0.25, 0.0 as values. We plot the the organisational constraint (blue) and the resulting production (red). As the organisational constraint is independent from the meta-parameters the blue line will be the same in all sub-graphs. Let us concentrate on both simulations with a low learning rate for the member agents (i.e., = 0.001 and = 0.002 on the left and in the middle). For all values of one can observe the same trend: Whenever the impact of the member agents is small ( ≈ 1) we see that the resource production follows the organisational constraints very closely; whenever the impact of the organisation decreases (and member agents’ impact increases) then the production frequency converges to the average (0.6 + 0.1)/2 = 0.35 of the two states since the agents could not adapt to the context change fast enough.
Since the organisation expresses the desired production, we obtain the result that for a less dominant organisation (smaller ) the system’s performance (measured as the fit to the desired resource mix) decreases.
Whenever the organisation is less dominant it is interesting to study whether a higher transparency might compensate for this. We simulate our scenario with a third learning rate = 0.01, one magnitude bigger than the ones before. In this case the agents will follow the 70 corg = 0.0 50 30 10 corg = 0.25 5700 30 10 70 corg = 0.5 50 30 10 desired organisational dynamics almost immediately and a diagram analogously to Figure 5 will show indistinguishable curves. We study the impact of the organisational constraints on resource production for this scenario, too (cf. right column of Fig. 6). We can see that his high learning rate is able to compensate a less assertive organisation since the production follows the desired curve even if the organisational impact is set to = 0. One can see that for a rate of = 0.01 an organisation with no impact (i.e., = 0) is comparable to an organisation with a very high impact but only a small learning rate = 0.001 – as the diagram for = 0.01 and = 0 looks roughly like that for = 0.001 and = 0.75.
From the simulation results we can identify those configurations with a high quality impact: They have either a high value for or for – or both. We can also identify interchangability as an increase of the transparency can compensate a decrease of organisational impact. In this simple scenario we cannot observe indispensability at all.
1 2 3 4 5 6 7 8 = 0.01 S1 In scenario S1 we still oscillate between two states, but the states require some symmetry breaking, i.e., some coordination between agents, since the desired outcome is either a rather low or a rather high frequency for the production of resource . In other words: The agents should make the same decision. Since the WFN does not provide any coordination here, the only source of coordination is the organisation itself. The organisation constraints this situation to the low frequency decision.
The OMA for 1 starts with a selection preference for resource of 1 = 0.7, which is likely to stabilise – in both states – in the left minima of Figure 4 (b), while the OMA for 2 starts with 2 = 0.4, which is likely to stabilise on the right side. Therefore, we expect that a gradient descent will lead the OMAs to diferent values.
The organisation is intended to reduce the price of anarchy by constraining the situation. Here, we set the organisational constraint to = 0.6 for the first state, which in between the minima of this state, and to = 0.2, which is the left minimum, for the second.
Similarly to S0 we can identify configurations in Figure 7 with a high quality impact. One can observe that a system without organisational control (i.e., for small , which are shown at the top) does not follow the desired production profile at all. For all values of the learning rate we see that increasing the organisational impact (i.e., going down in the figure) leads to an ever closer behaviour. Additionally, we can observe that for a fixed and a small learning rate a change of shows only insignificant efects, while a small increase of really improves the quality. For very high values of one can observe the phenomenon of over-shooting for gradient descent. Learning at high-rates introduces some kind of instability.
Remarkably, unlike in S0 we see indispensability for S1 in Fig. 7: For small values of we have configurations where the learning step size has almost no impact, i.e., we cannot trade organisational impact for an increased learning activity of the member agents.
The fact that a lower organisational impact could not be compensated by a higher learning activity indicates that organisational coordination is essential for S1 and the designer should not lower the impact value below a certain value.
1 2 3 4 5 6 7 8 ctra = 0.0 1 2 3 4 5 6 7 8 ctra = 0.001 1 2 3 4 5 6 7 8 ctra = 0.01 S2 Scenario S2 is more complicated since we vary the organisational impact against diferent transparency values and for diferent transparencies for a fixed . Since we use our scenarios to demonstrate the concepts we will restrict ourselves to only one value for . 60 corg = 0.25 40 corg = 0.5
60 corg = 0.75 40 20 60 40 20 20 p_Org res A
In Figure 8 one can observe interchangability: a lesser organisational impact (i.e., going up in the figure) can be compensated by learning provided that the organisational constraints are transparent to the learning member agents (i.e., by going right in the figure). We can observe a kind of fixed exchange-rate of learning/transparency and organisational constraints as the diagram does not change much when going one step up and one step right at the same time.
A more detailed evaluation for diferent values of (not shown here) also reveals the property of visibility: A lower organisational impact could not be compensated by a higher learning activity alone; it is necessary that the organisation additionally sends a strong transparency signal.
S3 In this scenario we assume the same rewards as for S1 and S2 (cf. Figure 4 (b)): For the first state we have minima at − = 0.5 and + = 0.7 and for the second state we have − = 0.2 and + = 0.8. In S1/S2 the organisation constraints the OMAs in the teams to = 0.6 for the ifrst state and to = 0.2 for the second. As a diference to S1/S2, we have an organisation that is not well adjusted, i.e., has not adapted to the recent environmental changes yet. Concretely, we set the organisation constraints to = 0.6 (just in between the minima) for the first state and to = 0.0 (which is too low) for the second.
Note, that in S3 the desired production is unchanged but the blue line for is now below that; e.g., for = 0.5 we see that the smaller transparency value = 0.001 is better than the greater one = 0.1 since the production is closer to the desired outcome of 0.2. 1 2 3 4 5 6 7 8 ctra = 0.0 1 2 3 4 5 6 7 8 ctra = 0.001 1 2 3 4 5 6 7 8 ctra = 0.01
For the previous scenarios S0-S2 we could identify the quality impact by the rule of thumb: The more, the better. This is no longer true, as can be seen in Figure 9. It shows the frequency for a fixed learning rate = 0.1, but for diferent values of and . Here, one can observe that whenever the organisational constraints are ‘wrong’ somehow, then transparency value above a certain value (here: the right column of Fig. 9) is contra-productive (i.e., we corg = 0.25 45 25 5 corg = 0.5 45 25 5 65 65 65 corg = 0.75 45 25 5 p_Org res A seems at least plausible, since one might expect a good compromise around a ‘middle’ value of ≈ 0.5. Here, we see that the ‘compromise area’ is in fact a little bit higher: ≈ 0.6. For the transparency value we observe that quite moderate values are already suficient.
Interestingly, we also have a ‘second-best’ cluster at = 0.01 and ≤ 0.2, i.e., for a high learning impact , that is also a good candidate when combined with small values of organisational impact . This seems to be counter-intuitive, but for S3 the organisational constraint doesn’t match the environmental signals well and therefore it is maybe a good idea not to follow the organisation too much and better invest in learning activities. Therefore, the good configurations of this scenario roughly follow a curve where the product of learning impact and transparency remains constant. Of course, the concrete reason why a combination of the meta-parameters results in a good performance requires an in-depth analysis of the model’s behaviour.
We studied the planning phase of a MAPE-loop for Org-MAS where we use a digital twin to predict the benefit of adaptations of the organisation model at run-time. The demand for a recursive formalism together with the fact that Sonar is a Petri net based formalism makes nets-within-nets highly interesting for our purpose here.
In this paper we studied the impact of the meta-parameters of the digital twin. We introduced properties of the organisational behaviour with respect to our meta-parameters (like quality impact, interchangeability, indispensability, visibility, and cumbersomeness) and studied the for example scenarios of a coordinated production.
The twin model gives us the opportunity for quantitative planning of adaptations, i.e. the Sonar-MAPE-Loop can decide whether a possible adaptation results into an organisation model that will perform better than the current one (i.e., we evaluate the benefit) and whether the amount of transformations (i.e., the costs) will pay of in the long run.
We have also seen that our digital twin model can be used to improve the deployment of the Org-MAS. If we find out that another setting of the meta-parameters is more beneficial, we can try to change the implementation in such a way that it is an counterpart of this better setting.
At the time being, we make an ‘educated guess’ how a concrete Org-MAS architecture has to be translated into a meta-parameter setting of the digital twin. Especially, we concentrate on our so-called teamwork parameters as mentioned above. In the case of a parameter sweep we are faced with the opposite problem, i.e., we have to translate a good setting of the metaparameters into concrete deployment details of the Org-MAS. In current work, we are deepening our understanding which design choices of our concrete applications are expressed by which parameter settings.
Additionally, we would like to integrate a feedback into these settings: We will monitor the real system for a while and fit the meta-parameters according to our observations. Then we collect some more data, adjust the parameters, collect some more data, and so on. Of course, this a kind of chicken-and-egg situation and we have to start somewhere without any prior observation data. So a good understanding of the relationship between the real architecture and the meta-parameters remains essential.
The Renew-Model of the complete Twin of our Sonar-MAPE-Loop is quite large and consists of several components. These components are connected using the Renew-mechanism of synchronisation channels. The overall view is shown in Figure 11. In the following we present a ‘zoom’ into the Petri net model, presenting all the model’s parts.
The source files are available at: https://github.com/koehler-bussmeier/digitaltwin/. The Renew simulator is available at: https://www.informatik.uni-hamburg.de/TGI/renew/.
The core of the Sonar-engine is defined by the general execution loop (cf. Fig. 12). This net defines the overall process of the Sonar-MAPE loop, i.e., the formation of a team and the accompanied adaptation activities, i.e., adapt to change of and adapt configuration .
Its main activity, the formation of teams, is started by the transition sense (OPA). It has an empty preset, as it is activated by the environment. Here, a task triggers the formation of a team of agents. The transition trigger teamwork takes this sensor state (i.e. a trigger) and generates a team-wfn, i.e., a pair of a team and a workflow net (WFN) to handle the task that is triggered. The transition has the places role-wfn, orga protocols and SONAR organisation network model as side conditions, which are used as a ‘database’ to make a look-up. The organisation model is a token on the place Sonar organisation network model. Each WFN from the set is a net-token on the place role wfn/protocols.
The Position agent model is shown in Fig. 13. The main purpose of this net is to trigger the production teamwork, which is then handled by the organisation net. It also contains the logic that the environment oscillates each = 500 production steps between the two diferent states, which have a diferent reward signal for the production (cf. Figure 4 (b)).
The Workflow with Roles 1 and 2 The workflow net (i.e. the interaction protocol) is shown in Fig. 14. It defines the interaction of two roles: 1 and 2 The left part of belongs to 1 and defines [1]; the right part constitutes [2]. In a team agents will be assigned to these roles.
Each role models a producer which has the choice between two possible goods: or . The Renew-model generates a random number (calling Math.random()) to solve the conflict between and . An agent implementing one of these roles has a preference order on these goods, modelled as a probability over and . Here, 2 denotes the probability that the agent will choose to produce . Additionally, the organisation defines a constraint on the production in the team formation; here, 1 denotes the organisational constraint to produce . Both probabilities are combined according to (1):
= · OPA + (1 − ) · OMA In the net, the decision conflict is resolved randomly by the inscription Math.random() < p.
The production steps of both roles are executed for = 500 steps; the upper right part contains a counter place for the number of steps. After each step both agents will adapt their decision probabilities over the channel this:adapt.
The organisation net of Fig. 15. reacts upon exactly one task, which initiates the production of resources and . We have three OPAs: 0, 1 and 2.
Whenever a task triggers a team-formation, the team is generated as an ongoing interaction between the MAPE-loop and an instance (i.e., the team) of the organisation net. This interaction takes place using the transition split, delegate, refine, assign exec.
The inner state of the team decides which concrete team construction steps are enabled Usually, we have more than one enabled transition, which, in general, leads to a combinatoric explosion of possible teams that could be generated by the organisation for a given trigger/task.
During this creation of teams the agents will be assigned to the roles of the team workflow, since, sooner or later, each of these formation processes must end with several firings of transitions named assign, drawn at the bottom of the organisation net.
The given organisation model can create exactly two teams for handling this task – one for each state of the environment. The main decision during the team formation (i.e. the decision which of the teams is chosen) is made by 0. For both possible teams the agent 1 is responsible for role 1 and 2 for 2. The teams difer in the organisational constraints imposed on 1 and 2. The OPA 0 adapts to the external change by switching between the two teams. Essentially this means that the production probability of good is = 0.6 for the first state and = 0.1 for the second. The member agent net is given in Fig. 16. Its main purpose is to implement the member agents’ learning process as defined in (2): ( + 1) = (1 − ) · ︁( () − · ′(())︁)
The two two transitions named g are used to compute the current gradient. We have two transitions here to handel the diferent parameters of the reward functions that we used to model the two states between which the environment oscillates.
For the exploration of design candidates (as done in Section 7) we generate a simulation run for each configuration of the meta-parameters – as given in the initial marking.
We use another Petri net to manage this batch of experiments (shown in Fig. 18). Here, we evaluate all combinations for a parameter sweep for all combinations of = 0.1 . . . , 0.9 and = 0, 0.0001, 0.001, 0, 01, 0.1 – once for = 0.01 and then for = 0.1. For each of these configurations a fresh instance of the simulation is started and evaluated.