=Paper=
{{Paper
|id=Vol-2114/paper10
|storemode=property
|title=A Policy Design Framework Using Agent Based Social Simulations
|pdfUrl=https://ceur-ws.org/Vol-2114/paper10.pdf
|volume=Vol-2114
|authors=Alexander Melchior
}}
==A Policy Design Framework Using Agent Based Social Simulations==
https://ceur-ws.org/Vol-2114/paper10.pdf
A Policy Design Framework Using Agent-Based
Social Simulations
Alexander Melchior
Department of Information and Computer Science, Utrecht University
A.T.Melchior@uu.nl
Abstract. In this paper we propose the development of a policy design
framework using agent-based social simulations (ABSS). ABSS enable us
to clearly articulate the problem(s) that we want to solve with policies
and simulate the proposed policies to remedy the problem. One impor-
tant aspect that we see in current policy development is the lack of
awareness regarding context, perspective and bias. When a policy is de-
veloped, for instance, within one ministry it will usually have the context
of that ministry, ignoring other perspectives in which the problem can
be seen. This causes a (unintended) bias during the policy development
and in the resulting policy. By creating an ABSS multiple contexts and
perspective can be incorporated into one model to explicitly articulate
these perspectives and biases. It also enables policy developers with ex-
pertise in a single context to add their insights to the model and see how
this interacts with other contexts during the simulation. The emerging
behavior in the ABSS gives insight in the different policy contexts.
The proposed framework enables policy makers to match their policy
problem with different ABSS design methods to see which best fits their
needs. At the same time social simulation experts can gain more insight
in the applicability of ABSS to policies.
Keywords: Policy development, Social simulations, Agent-based social
simulations, Emerging behaviour, Simulation comparison, Model, Frame-
work
1 Introduction
Policy making, in all its forms, is extremely complex. This realization is re-
peatedly discovered, something that was already pointed out a few decades ago
[1]. Developing policies is often done without clear goals, stakeholders, ideal
processes, (evaluation) metrics or open mindedness to new ways of working, ac-
cording to academics and governmental reports [1,2,3]. Not only is it complex,
it is also done in a dynamic world where political or societal surges in attention
can cause unexpected pivoting of the policy development direction. One way to
deal with this complexity is to use models in order to simplify problems, analyze
them and predict the effect of policies. In the past decade more types of mod-
els and approaches to policy making have become computational feasible, thus
giving us more options to deal with the policy design complexity.
One of the new approaches that seems to be a promising way to deal with
the complexity is agent-based social simulation (ABSS) [4,5,6,7,8,9]. Creating
an ABSS enables us to articulate a problem, make it explicit and look at it from
many different angles. By doing so we expect to be able to improve the policy
making design process. The applicability of this idea is illustrated by different
examples [10,11,12] from different perspectives and disciplines.
While we see the use of agent-based social simulations in these different use-
cases we see no general approach for policy makers to use ABSS as design tool.
A framework that uses ABSS in a policy design process enables policy makers to
use this new approach in their work. The framework can also help the academic
research field of social simulations, that is shared by many different disciplines, to
further the understanding of ABSS and their application to policy development.
This research project is sponsored by the Dutch Ministry of Economic Affairs and
Climate Policy. As such we will focus on (socio-)economic and climate related
policy development.
In this paper we will first describe the problem more thoroughly, define the
problem domain and research field. We will reflect on the state of the art and
the related literature. After this the research question and research method is
considered together with the means to accomplish our goals. We conclude with
the proposed approach for this research project and a short discussion on our
expected contribution.
2 Problem Statement and Related Work
In the introduction we touched upon the complexity of policy making and the
use of models to deal with this complexity. The ability to make models of reality
is very powerful as it enables us to come up with solutions for problems that
would otherwise be too complex or big to comprehend. But the use of models
not only brings us bliss, it has its limitations as well. Models are created in the
context of a perceived problem that manifests itself in reality; thus the model
has a perspective, a point of view, a bias, a context. While the initial creators
of the model mostly know the context of a model this context is often forgotten
or ignored by other users. Especially when a model seems to be working well in
the original context the attention for the context of the model is quickly lost.
The importance of context is easy to overlook when using seemingly good
models, but will result in situations as shown in Fig.1. Here the initial model
is created to take different aspects of problem A into account such that the
generalizations in the model match the context of the problem. Once this model
produces good results other users adopt it for their own problems. Only a part of
the modeled aspects apply to problem B, resulting in a weak connection between
the model and reality.
One of the reasons for this is the difficulty in having a good insight in the
context, limitations and scope of a model. While policy development frameworks
like the Dutch integrated impact assessment framework (IAK) [13] are used in the
Netherlands to guide the general policy development process it is still hard to see
83
Fig. 1. A model with a strong or weak connection to reality depending on the context
the actual problems deep down in the used models to make such assessments.
One could take the example of macro-economic models. These are based on
micro-economic models that are based on rational agents. Unfortunately the
core assumptions of rational agents can be questioned [8].
The existence and the challenge of this complex, context dependent, biased,
world of models is voiced by academics and policy makers more and more often
[14,6,15,9]. The current “state of the art” solution is the thought that more
research needs to be done in the area of using ABSS for policy development [7].
In ABSS one is able to explicitly model goals and means of agents, rep-
resenting humans. This could include interaction with other agents, changing
preferences of an agent, ways of reasoning of an agent and much more. Depend-
ing on the context an agent can have different goals and means. This enables
policy makers to add elements to the simulation given their own context. Other
policy makers can add their insights, from a different context, to the same model.
When we run the simulation the interaction between the different contexts can
be analyzed by studing the agent‘s behavior.
2.1 Problem Domain, Research Field and Literature
Problem domain: Policy development We define our problem domain as
“policy development”, something that we have describe up to now as a complex
process. While policy development cycles and other approaches are defined in
literature, real policy development isn’t as nicely structured. Yet some trends are
visible despite the fact that each policy department (within Dutch ministries)
has its own way of making policies. One trend that we see is the increased focus
on quantitative analysis and evaluation, see [16] for an example from economics.
Designing an ABSS forces a policy developer to articulate the problem and
quantify it, thus improving the potential evaluation and the use of quantitative
data. Another trend is the digitalization of society and government.
84
Research field: The social simulation community The idea of using ABSS
for policy development is not a new one, but there has been no structured work
on a general approach to use ABSS as a policy development tool. As such we
identify the need for a framework that uses ABSS as a policy development tool.
Others in the field of ABSS have identified this need as well [17]. Just like policy
makers, the academics working with ABSS come from many different research
fields. They come from, and this is very limited set, fields like computer science,
economics, psychology, social science and philosophy. Each field works in a very
different context and has different biases and goals. Yet all of them see ABSS
as a promising topic. A framework for using ABSS for policy development could
improve the understanding of other ABSS research and help the maturing of
this research field.
This is also represented in the literature that should give us a better under-
standing of the current state of the art. As discussed the current status seems
to be that the need for research in this direction is needed [4,7,9] and oth-
ers are proposing similar approaches [17]. This could answer the need for new
ways towards policy development [16,2,18,15] Recent published works such as
“Simulating Social Complexity: A Handbook” [19,6] enables us to get quickly
acquainted with the current techniques. Modeling and context importance is
treated in [20,14] among others. Examples of ABSS like [21] can be further stud-
ied as well.
3 Research Question
Our main goal is to build a framework for policy development using agent-based
social simulation design principles, this is reflected in RQ2. This framework will
require a way to compare possible solutions to be able to say something about
a proposed design, as stated in RQ3. Through answering RQ1 we will gain the
knowledge and tools needed to answer RQ2 and RQ3.
In Section 4 we will use the research questions to formulate our proposed
approach for this research project in order to answer the research questions.
– RQ1: What does the current policy design world look like? (Knowledge prob-
lem)
• RQ1.1: How does the literature support the problem statement? (Knowl-
edge problem)
• RQ1.2: What would the goal be of a policy design framework? (Knowl-
edge problem)
• RQ1.3: Can we identify a useful use case for explorative work and vali-
dation? (Knowledge problem)
– RQ2: What does a framework for the design of agent-based social simulations
for policy development look like? (Design problem)
• RQ2.1: How can we characterize a problem that can be solved with
ABSS? (Design problem)
• RQ2.2: How can we characterize a type of ABSS design? (Design prob-
lem)
85
• RQ2.3: Can we match the characteristics of a problem with the charac-
teristics of an ABSS in an effective way? (Design problem)
– RQ3: How can we compare ABSS’s? (Design Problem)
• RQ3.1: How are ABSS evaluated? (Knowledge Problem)
• RQ3.2: Is a general measure possible for the quality of an ABSS? (Knowl-
edge Problem)
• RQ3.3: Is a richer ABSS a better ABSS? (Knowledge Problem)
– RQ4: Does the policy design framework improve the policy design process?
(Knowledge Problem)
• RQ4.1: What is the stakeholders’ perception when using the policy design
framework? (Knowledge Problem)
• RQ4.2: Are context, bias, and perspective made more explicit by using
the policy design framework? (Knowledge Problem)
• RQ4.3: How do developed policies using the policy design framework
compare to other policies? (Knowledge Problem)
Words such as effective, measure, quality and richer are all representations
of the word utility. At this stage of the research project we know that these
words are not accurate enough but also regard a straightforward interpretation
far from trivial. Once we have a better understanding of the problem we will use
more appropriate terms for them.
3.1 Research Method
For this research we conduct a design science project [22]. As such the research
questions are categorized in knowledge questions (where we answer questions
by looking at artifacts in a problem context) and design questions (where one
designs an artifact to improve the problem context).
The design science engineering cycle in Fig.2 guides us in placing our research
questions in the appropriate phases. We can place RQ1 and RQ3 in the problem
investigation, RQ2 and RQ3 in the Treatment design, RQ4 in the Treatment
validation phase. This guides us into the next Section (4) where we will discuss
the approach of this research project using the design cycle.
4 Proposed Approach and Preliminary Ideas
Phase 1: Problem Investigation. In our stakeholder analysis we have identi-
fied policy makers as main stakeholders. Their goal is to develop improved poli-
cies in order to satisfy their own stakeholders, which can be numerous. At the
same time some of these stakeholders can be directly involved with this research
project: government management, operations management, policy implementers
or politicians.
Another set of stakeholders are academics. They can further their insights
in the applicability of ABSS for policy development and the comparability of
ABSS.
86
Fig. 2. Engineering cycle taken from [22]. The question marks indicate knowledge
questions, and the exclamation marks indicate design problems.
We first plan to increase our understanding of the goals and needs of the
stakeholders with two courses of action. First we continue our literature research
on the following topics:
– Policy development
– Comparing problems and solutions
– ABSS design
– Framework design
But literature can not teach us everything. Next to literature review we will
construct an ABSS toy case with Repast. This will be more informative as we
get to experience what it actually means to build an ABSS. This also means
that a deeper understanding of policy development is needed if we wish to apply
the ABSS to policy development.
Phase 2: Treatment Design. During phase 1 of the research project we have
started with some requirement gathering by learning more about the problems
we face. We also expect to have learned more about the available treatments
that might be applicable to our problem statement and use these to create our
own policy design framework. The work in phase 1 will identify what the most
interesting initial approach will be for the creation of the framework, RQ2. While
working on RQ2 we will gather more information and insight for RQ3. RQ3 has
a major impact on the effectiveness of the solutions made for RQ2.
To make the requirements more concrete we will select a use case with policy
makers to use for the validation (phase 3).
Phase 3: Treatment Validation. We will test and validate the framework
by designing a policy based on the use-case found during phase 1 and 2 using
the policy design framework. The resulting policy and design process will be
87
evaluated with the policy making stakeholders and will provide input for RQ4.
With this feedback we expect to be able to validate and improve the framework,
as a good design cycle would do. The questions posed in RQ4 are still imprecise,
we expect to learn what the important questions are during phase 1 and 2.
We do like to note that the three phases are part of the engineering cycle,
as such we expect phase 3 to be input for a new phase 1 in this project and
continue to cycle through these phases.
4.1 Preliminary Ideas
Now that we have recounted our approach for this research project we can dis-
cuss our preliminary ideas for a framework. In Fig.3 we can see a possible rep-
resentation of the characteristics of a problem (or policy) and an ABSS (design
process). Each of these have n aspects, approaches or characteristics that can be
matched. The aspects can be modeled using ontological modeling which gives
us a structured but flexible way of modeling. Each of the ontologies has its own
context and creators, making it a social constructs. In the matching process we
can compare (RQ3) a match with other matches and select the matching set
that would be most useful for a problem.
Fig. 3. Example of aspects of a problem that can be matched to certain design ap-
proaches. Each of these can have its own context.
To compare matchings we can use approaches like TraceME [23] for soft-
ware comparisons (∆-measurement). The analogy here is that both software
and policies describe rules and interactions, especially if we represent a policy
in an ABSS. With the TraceME approach we might also be able to answer RQ3
and its subquestions, but this is something we will discover during the research
project.
For the use-case we are working with the ministry to find an interesting topic
with a focus group. We will use an action research [22,24] approach for the use-
case. Once a use-case is found and the development of the ABSS has started
the plan is to work in a short cyclic manner to be able to adjust the direction
of development quickly. This short cyclic way of working might be one of the
aspects of the framework: if one finds a new aspect of the problem due to the
design work done on the ABSS it might be best to choose a different design
approach for this new problem aspect.
88
4.2 Means
In order to bring this research project to a successful end we have a number of
means that are available to us. A hybrid civic servant and PhD-researcher posi-
tion: my positions both at the Ministry of Economic Affairs and Climate Policy
and Utrecht University enables us to easily use the networks of relevant experts
in both the policy development world and the academic world. Supervisor ex-
pertise: the experience of Marcela Ruiz in ∆-analysis and the expertise of Frank
Dignum in inter-disciplinary social simulations. Diverse personal background:
while my main background is in computer science I also have a background in
basic philosophy, social geography, urban planning and multiple advisory and
management roles. This mix matches the diversity of the ABSS and policy de-
sign field.
5 Conclusion
In this paper we have elaborated on our current understanding of one of the
problems at hand in the policy development world. ABSS seems to be well suited
to deal with context dependency issues of policy development and as such we
propose to use ABSS as a policy design framework. With this contribution we
expect to improve the policy development process and the quality of policies.
At the same time the framework can be used by other academics to gain more
insight in the applicability for policy development for their own work on ABSS.
This research project fits well in the current maturing process of the field of
social simulations as it focuses on the applicability and takes a step back to look
at the field of social simulations. I also like to thank my supervisors, Dr. Marcela
Ruiz and Dr. Frank Dignum, for their continued advice and expertise.
References
1. Bovens, M., Hart, P.: Understanding Policy Fiascoes. Transaction Publishers
(1998)
2. Studiegroep Informatiesamenleving en Overheid: Maak waar! (4 2017)
3. Nowak, A., Kacprzyk-Murawska, M., Serwotka, E. In: Social Psychology and the
Narrative Economy. Springer International Publishing, Cham (2017) 45–58
4. Axelrod, R.: Advancing the art of simulation in the social sciences. In Conte, R.,
Hegselmann, R., Terna, P., eds.: Simulating Social Phenomena, Berlin, Heidelberg,
Springer Berlin Heidelberg (1997) 21–40
5. Edmonds, B., Gershenson, C.: Modelling complexity for policy: Opportunities and
challenges. CoRR abs/1310.2290 (2013)
6. Edmonds, B., Meyer, R. In: Introduction. Springer International Publishing, Cham
(2017) 3–12
7. Gilbert, N., Ahrweiler, P., Barbrook-Johnson, P., Narasimhan, K.P., Wilkinson,
H.: Computational modelling of public policy: Reflections on practice. Journal of
Artificial Societies and Social Simulation 21(1) (2018) 14
8. Ormerod, P. In: Economics. Springer International Publishing, Cham (2017) 19–44
89
9. Rosewell, B. In: Complexity Science and the Art of Policy Making. Springer In-
ternational Publishing, Cham (2017) 159–178
10. Ghorbani, A., Dechesne, F., Dignum, V., Jonker, C.: Enhancing abm into an
inevitable tool for policy analysis. Policy and Complex Systems 1(1) (2014)
11. Maglio, P.P., Cefkin, M., Haas, P.J., Selinger, P.: Social factors in creating an
integrated capability for health system modeling and simulation. In Chai, S.K.,
Salerno, J.J., Mabry, P.L., eds.: Advances in Social Computing, Berlin, Heidelberg,
Springer Berlin Heidelberg (2010) 44–51
12. Montanola-Sales, C., Onggo, B.S., Casanovas-Garcia, J.: Agent-based simulation
validation: A case study in demographic simulation. In: Proceedings of the Third
International Conference on Advances in System Simulation (SIMUL 2011). (2011)
109–115
13. Dutch Ministry of Justice and Security: The integrated impact assessment frame-
work for policy and legislation (12 2016)
14. Edmonds, B. In: The Room Around the Elephant: Tackling Context-Dependency
in the Social Sciences. Springer International Publishing, Cham (2017) 195–208
15. Houppermans, M.: Twee Kanten van de Medaille: een onderzoek naar de kwaliteit
van de beleidsvoorbereiding. PhD thesis, U (September 2011)
16. Expertwerkgroep Effectmeting: Durf te meten - eindrapport expertwerkgroep ef-
fectmeting (11 2012)
17. Perello-Moragues, A., Noriega, P.: Using agent-based simulation to understand the
role of values in policy-making. SSC 2018 Conference, Stockholm (8 2018) Used
first submitted version.
18. Gilbert, N., Terna, P.: How to build and use agent-based models in social science.
Mind & Society 1(1) (Mar 2000) 57–72
19. Edmonds, B. In: Different Modelling Purposes. Springer International Publishing,
Cham (2017) 39–58
20. Edmonds, B.: The use of models - making mabs more informative. In Moss, S.,
Davidsson, P., eds.: Multi-Agent-Based Simulation, Berlin, Heidelberg, Springer
Berlin Heidelberg (2001) 15–32
21. Ahrweiler, P.: Agent-based simulation for science, technology, and innovation pol-
icy. Scientometrics 110(1) (Jan 2017) 391–415
22. Wieringa, R.: Design science methodology for information systems and software
engineering. Springer (2014) 10.1007/978-3-662-43839-8.
23. Carmona, L.M.R.: TraceME: Traceability-based Method for Conceptual Model
Evolution. PhD thesis, Universitat Politècnica de València (2016)
24. Davison, R., Martinsons, M.G., Kock, N.: Principles of canonical action research.
Information systems journal 14(1) (2004) 65–86
90