The research of a balance between the growing pressure of addressing long-term sustainability issues and the existence of short-term economic and political challenges is one of the main issues of this century. To contribute to this case, we design a four-players game to elicit this tension. Then, an agent-based model of this game is developed, to observe the efects and the efectiveness of various behaviours and strategies, especially regarding risk preferences. An evolutionary meta-model selects the best-performing agents during multiple generations, and data from the resulting population are collected. The analyses of the results suggest that environmental factors afect the resulting risk sensitivities.
The concept of sustainability was present in human societies much before the term became
popular. In the middle age, policymakers of the Republic of Venice were already concerned
about the scarcity of wood related to the consumption of the Venetian forests, which could
preclude the production of new ships crucial for the city’s economy. Hence, they impose some
limitations on employing this resource for private purposes in selected areas to enable forest
renovation and not consume all the life-stock [
There is ample literature documenting the urgent sustainability problems facing the modern
world [
RPG/Video game
Online sandbox game Computer simulation game
Examples Learning SD (LSD)
Samba Role Play Industrial Chlorine Transport
Metagame
Keep Cool River Basin Game
Atollgame
MHP
About That Forest Evacuation Challenge Game The Arcade Wire: Oil God 3rd World Farmer Climate Challenge
Stop Disasters!
Energyville
Encon City New Shores: A Game for Democracy
World Climate
Green City Tragedy of the Tuna The UVA Bay Game
Aims and themes Env. Conservation and urban
development
Natural resource management Social technology as a SD public policy
Sustainable process negotiation
Water cleaning
Groundwater cleaning Strategies for SD technologies
Sustainable forest management Natural disaster and emergency planning
Food sustainability and geopolitics Sustainability of agricultural land use Renewable energy sources and politics
Natural disasters prevention Sustainable energy supply
Energy conservation
Green project management Global warming decision-making
Green urban development
Water resources management Sustainable products and services cope with ever-increasing sustainability challenges [6]. The last decades have seen substantial growth in the literature on SSGs, with games varying from board games, to computer and web-assisted gaming, to role-playing or a mix of these elements [7].
Table 1 highlights some examples of serious games developed to facilitate sustainable goals [8].
In this paper, we propose a sustainability game that involves investments in either renewable, non-renewable or military sectors, we simulated it with intelligent agents, and we observe how evolutionary adaptation in a sustainability setting afects risk preferences.
This sustainability game can be mapped to a discrete dynamical system that allows us to explore its outcomes in a systematic way. The methodology utilised is dynamical system simulation, a discipline with a long tradition of being applied in sustainability settings at least since the pioneering model called World3 by Jay Forrester generates the data for the Limit to Growth report in 1972 [9]. More recently, both agent-based and system dynamics models have been built to face sustainable issues [10, 11].
The research addresses the intertwined relationship between sustainability, renewable resource stocks and risk preferences in production decisions. A risk preference is a property of every decision-making entity, and it defines how it addresses the uncertainty in the outcome of the decision. For example, one can assume to have two alternatives and with the same expected results, quantifiable in terms of utility: a risk-averse entity favour the option B (with lower variability) and a risk-prone entity prefer option A (with higher variability) [12]. An entity without a preference between and is called risk-neutral. Here, we refer to a non-risk neutral entity as "risk sensible" and the property of having a risk preference as "risk sensitivity".
This work aims at understanding how risk sensitivity adapts by genetic evolution in a setting with renewable resources and how this adaptation is afected by environmental factors, with attention to the dangerousness of the scenario. The results show that non-trivial relationships exist between these parameters and the resulting risk sensitivities. The generality of the setting suggests that the conclusions of this research could have some real-world implications regarding how diferent entities address risk in complex environments with competition and sustainability issues.
The paper divides as follows. Section 2 presents the methodology employed for the research, dividing it into the serious game, its agent-based model, and the evolutionary meta-model used to generate the results. Section 3 shows and discusses the results, while Section 4 draws some conclusions.
This section displays the methodologies employed in the research work and contains four parts: the sustainability game; its agent-based model; the evolutionary meta-model; and the experimental setting.
The game introduced in this paper aims to put players in a competitive environment where long-term sustainability goals contrast short-term production and defence objectives. In what follows, the game is described twice: firstly qualitatively, then employing diference equations (Equations 1-18).
The primary components of the game are four kinds of blocks, which stands for a type of resource available. The blocks divide by colours for greater clarity: black, green, red and brown. Respectively, each type of block represents the following real-world resource: • the black blocks stand for non-sustainable industrial capacity; • the green blocks stand for sustainable industrial capacity; • the red blocks stand for the military capacity; • the brown block stand for the biosphere.
The game is comprised of 800 turns, so each player plays 200 times. Each turn, a player can make decisions regarding two aspects: produce new blocks and use the red blocks to attack another participant. Each player begins the game with a single black block and no other blocks. Players have a certain integer amount of black, green and red blocks. The brown blocks, which stand for the stock of natural resources, are shared between all players. A player leaves the game when one of these four conditions is satisfied. First, it is the sole player remaining in the game. In that case, it wins. Second, the shared level of the brown blocks is zero. This condition embodies a circumstance in which human activities destroyed the biosphere. In this situation, all the players lose. Third, a player has no green or black blocks left. In that situation, it individually loses while the game proceeds. Fourth, at the end of turn 800, the player is still in the game. In that case, each participant still in the game wins. Intuitively, the game creates tension between individual short-term and global long-term goals.
The production blocks, which are the black and the green, can be used in the following way, to produce new blocks every turn.
• 1 black block can be use each turn to produce 1 black block; • 2 black blocks can be use each turn to produce 1 green block, destroying the black blocks; • 2 green block can be use each turn to produce 1 green block; • 2 green block can be use each turn to produce 1 black block, destroying the green blocks; • 1 black block can be use each turn to produce 1 red block; • 1 green block can be use each turn to produce 1 red block; • 2 green blocks can be use each turn to produce 1 brown block;
Each black and red block consume one brown block at each time step. In this way, the more non-sustainable industrial and military capability a player possesses, the more they destroys the natural stock. This rule has a single exception, which is that every green block can sustain a red block, not making it eroding the common stock of brown blocks. So, for instance, if a player has eight green blocks and ten red blocks, it consumes only two brown blocks per turn from red blocks maintenance.
The last rule regards the usage of the red blocks, which stands for the military capacity. Red blocks can be operated to fight another player, and each player can attack only once per turn. When the two players engage, each one loses a number of red blocks equal to the minimum number between their red blocks and the opponent’s red blocks. Then, if the attacking player holds more red blocks than the defending player, it takes other blocks from it. Firstly, it takes one black block per red block outlasted from the combat. Then, if the number of red blocks remaining is greater than the black blocks of the defending agents, it takes also a number of green blocks equal to the minimum amount between the remaining red blocks and the green blocks owned by the defending player.
Formally, given (), (), () respectively the number of black, green and red blocks owned by player at the beginning of the turn , and () the number of brown blocks remaining at the beginning of turn . (), (), (), (), (), (), () are the specific decisions of each player, in term of deciding which resource to produce another resource. is the attacked player, () a Boolean variable that define if the player attacks or not during a turn, and (), (), () respectively the number of black, green and red blocks owned by player at the beginning of the turn . Finally, () and () are the variation of each kind of block in player stock related to the blocks stolen from player in case of victory, and () is the amount of red blocks lost by player during a fight, and () the number of red blocks of the player surviving the fight. The stock variations for each kind of stock is outlined by the following equations: ( + 1) = () + () + 0.5 × () − () + ( + 1) = () + () + () − () +
( + 1) = () + () + () − ( + 1) = () + () − () − (() − (), 0) There are 7 production boundaries, one for each resource-production decision. ()() − () − () ()() − () − () ()() − () − () ()() − () − () − () ()() − () − () − () ()() − () − () − () (1) (2) (3) (4) (5) (6) (7) (8) (9) Lastly, the results of the fights are computed according to these boundaries and equations. ()() − () − () − () () = () × ((), ())
() = () − () () = () × ((), ()) () = () × ( ((), ()) , ()) ( + 1) = () − () ( + 1) = () − () ( + 1) = ( () − () × (), 0)
This section presents a simulation model of the game shown in the previous section. Equations 118 formalize the game dynamic and its boundaries for every turn. However, the most interesting element is the players’ decision-making related to the allocation of the resources (e.g., variables (), (), (), (), (), (), ()) and the aggression to other players (e.g., variables () and ). For this reason, the model is not simulated by means of a classic stock-flow model but using an agent-based technique, which suits the representation of individual behaviours and appraises their efect on the overall system [ 13]. The agent-based model includes a single kind of entity, which is the player. In a single game, there are 4 agents. Each simulation runs for 200 time steps, which means that each agent gets to decide 200 times. Figure 2 depicts the scheduling for each time step of the simulation model, which has three main phases: production phase, aggression phase and computing black blocks. The last stage is essential, consisting solely of two activities: computing the current number of brown blocks (see Equation 4) and terminating the game when the number of brown blocks is equal to or lower than zero. The other two phases are less trivial and embody agents’ decision-making.
Agents decide according to a tuple of 12 parameters, shown in Table 2. The parameters divide
into three categories. 1, 2, 3, 4, 5, 6, 7 influence the allocation of resources (e.g.,
the values of the variable bounded by Equations 5, 6, 7, 8, 9, 10 and 11). Hence, the higher one
of these parameters, the higher the possibility that one of the related input blocks (e.g., a black
block for 2) is employed to produce the related output block (e.g., a green block for 2). 1,
2, 3 afect the decision-making regarding which other agent to attack. This decision is
(11)
(12)
(13)
(14)
(15)
(16)
(17)
(18)
Propensity in producing black blocks from black blocks 1 ∈ [
Propensity in producing red blocks from black blocks 5 ∈ [
Propensity in producing red blocks from green blocks 6 ∈ [
Propensity of attacking a player with black blocks 1 ∈ [
Risk sensitivity regarding attack ∈ [
Intuitively, the decision-making process depicted in the previous section is not optimal and is not supposed to be. There could be many other ways to design and implement intelligent agents that play this game. Nevertheless, the scope of this study is not to identify the best possible strategy but to observe how the risk sensitivity evolves in repeated matches. This section depicts how we design and implement the evolutionary tournament.
Figure 3 presents the evolutionary meta-model, consisting of the following components. At the beginning of the meta-model, a population of agents (where is a parameter of the meta-model) is initialized. For each agent, the behavioural parameters presented in Table 3 are sampled from a uniform distribution. A uniform distribution is selected because all the allowed values vary between 0 and 1, and the sampled values do not cover diferent orders of magnitude. Then, the model process enters the first loop, consisting of three activities: creating random teams of agents, simulating the game and computing the score. For every match , agents gather into randomly-selected groups of 4 members, and each group enters into the agent-based simulation game described in the previous section. According to the round result, each agent collects a score in two ways: surviving until the end of the simulation (a shared victory) or being the sole agent to remain alive in a given intermediate time step (a lonely win). Each of these conditions provides a Boolean outcome and . The outcomes are weighted by a parameter , the importance of surviving until the end of the simulation, so that the score of a single match is = × + × (1 − ). if = 0, only the part related to victory (e.g., remains the only agent alive) matters, and if = 1, only staying alive until the end of the simulation is relevant to computing the final score. Later, the new score cumulates to the total score of the generation: () = ( − 1) + , with (0) = 0 The number of games per generation is a parameter of the evolutionary meta-model. When the number of matches played in a generation = , the generation ends, and the evolutionary selection starts. The evolutionary process creates a set of agents (the so-called evolutionary wheel), where each agent is replicated a number of times proportionally to its score. Every generation has a survival rate of (a parameter of the meta-model). It means that [(1 − )] agents dies during each selection process. Consequently, they need to be replaced with new [(1 − )] agents. So, then a random number of [(1 − )] agents is taken from the population and removed from the evolutionary meta-model. At the same time, an equal number of new agents is generated and added to the meta-model. Each new agent has two parents, randomly selected from the evolutionary wheel. So, the higher the score from the generation, the higher the probability of having ofspring. The behaviour parameters (see Table 3) of each new agent are the random combination of the behavioural parameters of the parents. Mutations occur with a rate , a parameter of the meta-model. After the evolutionary selection, there are two possible cases. If < , where is the number of the current generation, the previous stages repeat for a new generation. Contrarily, the meta-model is terminated.
Intuitively, the purpose of the evolutionary meta-model is to observe how the behavioural parameters of agents adapt under evolutionary pressure. In this research, we studied the two risk sensitivities parameters. In the next section, we present the experimental setting used to explore the evolutionary adaptation of this model.
The structure of this meta-model allowed us to perform experiments on two diferent groups of parameters: the parameters of the agent-based model and the parameter of the evolutionary meta-model. The experimental setting was designed to collect data on the efect of parameter variations on behavioural parameters (see Table 5), but for the purpose of this research, only the risk-sensitivity parameters are then analyzed. Also, experiments regarding the numerosity of the agents in the simulation did not provide any interesting results. So, they are not presented here. Figure 4 depicts the experimental setting with a black box diagram. For greater clarity, the input and output at the core of this research are bold.
An evolutionary meta-model run for every value of the input parameter of interest. Table 4 presents the four experiments performed with their target parameter variations. Table 5 shows the default configuration of the parameters when it is not specified diferently.
The agent-based model and the evolutionary meta-model were implemented using Python 3.9, without any specific framework for agent-based modelling or genetic algorithm, and every simulation run on a Windows machine equipped with a 3.30GHz Intel(R) Core(TM) i5-4590 CPU and 4.0 GB RAM.
This section presents and discusses the results of the experimental setting depicted in section 3.4. More precisely, it exhibits the efect of the variation of three diferent input parameters of the final risk sensitivities ( for resources and for actions) of the population of agents resulting from the evolutionary process. In the section, the same presentation structure is followed for each parameter. Four figures are displayed, illustrating the behaviour of the mean of risk sensitivities (two-line plots, respectively of and ) and the distribution of the two risk sensitivities for diferent values of the input parameter.
Figure 5 and Figure 6 show the presence of non-linear behaviour of the mean risk sensitivities of agents with the variation of the number of initial brown blocks. Figure 7 and Figure 8 confirm Figure 5 and Figure 6 by presenting the variation of their distribution with (0). This parameter is a proxy of the dangerousness of the environment. One can consider the game dynamic illustrated by Equations 1-18: a small number of brown blocks leads to a higher probability of an early endgame due to life-stock consumption. More precisely, since there were 4 players and a black block consumed 1 brown block per turn if all the players started the game by doubling the black blocks available, it took only 6 turns to consume all the stock of brown blocks. Oppositely, the higher (0), the longer the time available for deploying diferent strategies (for instance, attacking or converting to green blocks). Consequently, populations tend to be risk-neutral for (0) ≈ 0 whereas it was unlikely for an agent to survive. Therefore, the selection process is less eficient, resulting in "biased" populations. For higher values of (0), its efect difers for and . For the resource risk sensitivity , the higher the number of initial brown blocks, the more risk-seeking the resulting population. If the initial stock of the brown blocks was high more agents in the game could pursue a non-sustainable strategy without risking destroying the brown stock. Consequently, risk-seeking manners became more and more efective with the growth of the initial brown blocks until (0) got to a level over which it had no efect anymore for the reason that the initial stock of brown blocks was enough to consent the agents to survive without putting attention to the variations in the common stock. This result could have probably changed by increasing the number of turns of the game on the grounds that the longer the game proceeds, the higher the initial level of brown blocks needed to generate a more elevated . On the other hand, the attack risk sensitivity initially rose with the number of the initial brown blocks. The variation in the aggression expected return explains this result. When (0) is low, an addition in the value of the parameter made the selected population more favourable to risk because in that setting, the only way to get points more points than other players (and consequently, an evolutionary advantage) was to attack often and force them out of the games before the level of brown blocks got to 0 and everybody loses. Nevertheless, as soon (0) was high enough, risk-seeking behaviour regarding aggression got less and less fit by cause of the increased chances of surviving until the end of the game. So, it suited more to have a conservative approach and attack exclusively when the number of red blocks was high enough, not taking the chance of exposing to counterattacks and being kicked out of the game. This growth did not last for any values of (0). Figure 6 shows that when (0)12800, started to decrease, because the probability of survival of the population did not vary anymore with (0). Since = 0.5 (see Table 4), being a little bit more risk-seeking regarding the aggression led to an advantage: it permitted agents to win some matches that otherwise would have ended in a draw, taking away the possibility for other agents to increase their score, which was evolutionary advantageous.
Figure 9 and Figure 10 show the presence of non-linear behaviours of the risk sensitivities with the variation of the survival rate. This parameter stands for the harshness of selection in the evolutionary process. The lower , the smaller the amount [(1 − )] of agents that passed from one generation to another in the meta-model. It had a first implication, observable in Figure 11 and Figure 12, which is the variety in the dispersion of and in the populations. When < 0.8, the variability of the risk sensitivity was smaller. Since we utilised = 0.9 as the default value, Figure 7, Figure 8, Figure 15, and Figure 17 present wider probability distributions.
Figure 9 suggests that had a positive efect on . More precisely, the higher the share of agents surviving a generation, the higher the probability that an agent generated ofsprings even if ̸= 0 at each generation, since the number of chances grew with . To obtain ̸= 0, an agent had to survive until the endgame, but it was not necessary to be a sole winner. Therefore, the incentive to invest in non-renewable resources was lower and lower.
Figure 10 rather depicts a non-linear relationship between the and . The cause is similar to the one for (0). When was low, to attack or not attack does not substantially alter the probability that genes would be passed to ofspring considering a high number of agents are selected at each generation. Diferently, with the growth of fewer agents were selected on the evolutionary wheel every time. Then, a greater returns to be advantageous as in the default case. Finally, when ≈ 1, the majority of the agents survive to the next generation. Consequently, risk aversion regarding aggression became less and less advantageous as [(1 − )] is smaller, and a high value of is required to reproduce. Therefore, lower risk sensitivity meant a diminished probability of shared victories and a consequent increase in the reproducing chances.
Figure 13 and Figure 14 show the relationship between risk sensitivities and , which embodies the relevance in the evolutionary process of a shared or a lone victory: the higher , the more weight has to survive until the end of the simulation with other agents rather than being the sole agent to win the game.
Figure 13 and Figure 15 present a positive relationship between and , while Figure 14 and Figure 16 depict an analogue behaviour for and . In both cases, this result explains the efect of risk sensitivity parameters on the probabilities of shared or lone victory. More precisely, the linear relationship suggests that the more risk-seeking agents, the less likely were to get a shared victory. This conclusion derives from the growth of and the meta-model design. If afected and influenced the chance of reproducing, for ≈ 0 the values of and selected from the evolutionary process were the ones that maximize the probability of a sole victory. Diferently, with ≈ 0 the resulting and maximize the chances of a shared victory.
Interesting, for the variation of generates a change of sign, which is a variation in the overall behaviour. Figure 14 shows that the overall population of agents is risk-prone for < 0.4 and risk-averse for 0.4.
This paper presents an agent-based model of a simple sustainability game in which agents decide according to risk-preference parameters and an evolutionary meta-model. The research aims at showing how risk sensitivities evolve and how this evolution is afected by some environmental parameters such as the level of natural stock at the beginning of the simulated game (e.g., the brown blocks), the surviving rate per generation and the weight related to the kind of victory (lone victory or shared victory). The results show that multiple non-linear relationships exist between these parameters and the resulting risk sensitivities. While it is an abstract model, the conclusions can have real-world implications on how diferent entities address risk in complex environments.
The analysis focuses on the evolutionary adaptation of risk preferences. Nevertheless, the same simulation model could be employed to study diferent features. So, future research includes the analysis of the adaptation of other behavioural parameters, as well as the analysis of the co-efect of meta-model parameters on the output variables, for instance, through model exploration strategies such as grid sampling. Learning mechanisms could also be included in the meta-model so that agents do not only adapt by evolution but also by getting information from the environment and the actions of other agents. Finally, the model could be analysis also per se, and the efect of diferent strategies could be investigated, stand-alone or in relationship with other specific strategies. [6] H. Dieleman, D. Huisingh, Games by which to learn and teach about sustainable development: exploring the relevance of games and experiential learning for sustainability, Journal of cleaner production 14 (2006) 837–847. [7] P. Hallinger, R. Wang, C. Chatpinyakoop, V.-T. Nguyen, U.-P. Nguyen, A bibliometric review of research on simulations and serious games used in educating for sustainability, 1997–2019, Journal of Cleaner Production 256 (2020) 120358. [8] M. Stanitsas, K. Kirytopoulos, E. Vareilles, Facilitating sustainability transition through serious games: A systematic literature review, Journal of cleaner production 208 (2019) 924–936. [9] W. W. Meadows, Donella H; Meadows, Dennis L; Randers, Jørgen; Behrens III, The Limits to growth; A report for the Club of Rome’s project on the predicament of mankind, Universe Book, New York, 1972. URL: https://search.library.wisc.edu/catalog/999473210902121. [10] S. Pedell, L. Sterling, Agent-based modelling for understanding sustainability, in: Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 2011. doi:10.1007/978-3-642-25044-6_32. [11] N. C. Onat, M. Kucukvar, A. Halog, S. Cloutier, Systems thinking for life cycle sustainability assessment: A review of recent developments, applications, and future perspectives, 2017. doi:10.3390/su9050706. [12] S. Mishra, Decision-Making Under Risk: Integrating Perspectives From Biology, Economics, and Psychology, Personality and Social Psychology Review (2014). doi:10.1177/ 1088868314530517. [13] E. Bonabeau, Agent-based modeling: Methods and techniques for simulating human systems, Proceedings of the National Academy of Sciences of the United States of America (2002). doi:10.1073/pnas.082080899.