<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>Modelling e-participation implementation: A network-based approach for online and offline participation⋆</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Nicolas Bono Rossello</string-name>
          <email>nicolas.bonorossello@unamur.be</email>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Annick Castiaux</string-name>
          <email>annick.castiaux@unamur.be</email>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Anthony Simonofski</string-name>
          <email>anthony.simonofski@unamur.be</email>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>Namur Digital Institute, University of Namur</institution>
          ,
          <country country="BE">Belgium</country>
        </aff>
      </contrib-group>
      <pub-date>
        <year>2022</year>
      </pub-date>
      <fpage>0000</fpage>
      <lpage>0003</lpage>
      <abstract>
        <p>E-participation consists of several phases such as planning, implementation and evaluation. However, when representing this process, the implementation phase tends to be considered as a single block (the so-called "black-box"). This becomes a problem when the implementation combines ofiflne and online methods, as it requires a detailed characterization and representation of all elements involved. In this paper we tackle this issue by proposing a network-based model to describe these methods. This choice is motivated by the fact that network models allow to better describe the distributed nature of these activities. To build this model we make use of the theory in Social Networks Analysis (SNA) to represent the main interactions between all actors involved. To asses the reliability and added value of the presented model, this approach is applied to four different use cases that showcase various combinations of online and offline participation methods. The results of these use cases show the great potential of the network-based model as a tool for designing, comparing and evaluating different types of implementations. Namely, the visualization of the model allows to asses the level of participation, the role of the different actors and how different instruments are combined.</p>
      </abstract>
      <kwd-group>
        <kwd>eol&gt;E-participation</kwd>
        <kwd>Network-based modelling</kwd>
        <kwd>Participation methods</kwd>
      </kwd-group>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>1. Introduction</title>
      <p>
        E-participation aims at enabling and enhancing public participation through the use of technology.
In this context, the ultimate goal is to engage citizens in a communication and cooperation
process with the government. Various authors stress the fact that the use of Information and
Communication Technologies (ICT) does not provide a complete solution to the main participation
challenges [
        <xref ref-type="bibr" rid="ref1 ref2">1, 2</xref>
        ], and that the combination of e-participation with traditional ofiflne methods is
required to reach a broader audience and a higher rate of success. However, combining different
participation methods (online and ofiflne) is not undemanding, and to properly achieve this
aspiration a clear characterization of all the different elements involved in the implementation of
these methods is necessary.
      </p>
      <p>
        During the last years, there has been a strong effort in the e-participation field to formally
define the different phases of the participation process [
        <xref ref-type="bibr" rid="ref3">3</xref>
        ]. In general, the e-participation process
can be broadly divided into an initial phase of planning, an implementation phase and then an
evaluation phase [
        <xref ref-type="bibr" rid="ref3">3</xref>
        ].
      </p>
      <p>
        Recent works provide an overall picture of all the mentioned phases, the actors involved and
their relationships [
        <xref ref-type="bibr" rid="ref4 ref5">4, 5</xref>
        ]. These approaches are necessary so to obtain an extensive overview of
the e-participation process [
        <xref ref-type="bibr" rid="ref6">6</xref>
        ]. However, given the convoluted nature of this kind of processes,
these models produce very complex representations of all the elements involved in e-participation
which hinders their use during the development of the participation process.
      </p>
      <p>
        Among the aforementioned phases of e-participation, the planning and the implementation
can be seen in a sequential manner. The outcome from the planning and preparation phase
will influence the duration, goal and resources associated to the implementation. In that sense,
there are not that many studies focused exclusively on the modelling of the implementation part,
examining the different methods and their complementarity [
        <xref ref-type="bibr" rid="ref7">7</xref>
        ], which leads to the representation
of this phase as a single element (black box) in the context of large integrative models. This
absence of a more specific characterization has been observed as a possible limitation to the
application of e-participation by inexperienced practitioners [
        <xref ref-type="bibr" rid="ref8">8</xref>
        ] and developing countries [
        <xref ref-type="bibr" rid="ref9">9</xref>
        ]. In
these scenarios, detailed information regarding the implementation could help less experienced
practitioners to more effectively implement these participation methods.
      </p>
      <p>
        To this end, we propose a model that focuses on the description of the different methods
applied during the implementation phase. Given the aforementioned sequentiality of the process,
the use of a phase-specific model does not have a detrimental effect on the other phases or the
overall participation process. Thus, the design of a implementation-oriented model can be see
as an additional tool in a general and integrative approach rather than an alternative to these
models. In that regard, we propose the use of a network-based model to obtain this representation.
Network models are a common tool to describe social interactions where the main appeal lies in
the distribution and interconnection between the different agents [
        <xref ref-type="bibr" rid="ref10">10</xref>
        ]. This method allows us to
obtain a more detailed description, not only of the amount of actors and resources, but also their
roles and distribution.
      </p>
    </sec>
    <sec id="sec-2">
      <title>2. Related work: Modelling approaches to e-participation</title>
      <p>
        In her seminal paper about e-participation, Macintosh [
        <xref ref-type="bibr" rid="ref11">11</xref>
        ] refers to the need of properly
characterizing the different e-participation processes so to define a standard way to describe the actions
and outcomes of these activities. Since then, this formalization has been extended by
providing the relations between these concepts in the form of ontologies, metamodels and semantic
descriptions [
        <xref ref-type="bibr" rid="ref12 ref4 ref5">12, 4, 5</xref>
        ].
      </p>
      <p>
        Within this context, Porwol et al. [
        <xref ref-type="bibr" rid="ref5">5</xref>
        ] provide an integrated ontology for e-Participation
initiatives. Their work aims at producing a controlled and formal vocabulary for e-participation
demands, where they implement an integrative model that comprises the platform
conceptualization, the project conceptualization, and the democratic process. In the same line, Yusuf
et al. [
        <xref ref-type="bibr" rid="ref13">13</xref>
        ] provide an e-participation framework where several aspects concerning participants
and government complexity are discussed. Islam [
        <xref ref-type="bibr" rid="ref14">14</xref>
        ] develops a sustainable framework for the
development of an implementation model, but most aspects of the framework are theoretical
concepts envisioning a future implementation. As mentioned by Santamaria-Philco et al. [
        <xref ref-type="bibr" rid="ref3">3</xref>
        ],
these frameworks were mainly conceived as theoretical constructions not very suitable for
implementation. While frameworks and ontologies provide the overall conceptual structure of a process,
models explore the specific implementations [ 15]. Thus, these works constitute a solid foundation
to construct our model, as they define and characterize the main elements of participation, while
we provide a more practical focus to the representation of the process itself. By focusing on the
relations within the process, we can better analyze the differences and similarities between online
or offline approaches.
      </p>
      <p>The modelling of the main interactions during e-participation can help to better understand
the complex participation process [16]. Networks and agent-based models have been used in
several fields where there is a clear need to understand social interactions [ 17]. In the field of
citizen participation, agent-based modelling (ABM) has been used to model citizen participation
activities and to make predictions of the complex behaviour from citizens based on stochastic
simulations [18, 19]. To be able to evaluate the outcome of these simulations they randomly
define the attributes associated to each agent based on broad and general data collected from
interviews to citizens of the area. The main inconvenient of these approaches is that they rely
on approximations and random generation of some social elements, making them very valuable
in simulations to observe macroscopic behaviour but not suitable for descriptiveness of given
implementations.</p>
      <p>Social networks can also be used in a more descriptive manner. Kautz [16] uses complex
adaptive systems to describe the connections in distributed participatory methods. Piperagkas
et al. [20] modelled participation processes as a social participation network including implicit
actions like possible interest or possible collaboration, as well as contextual aspects like themes.
While the approach is very interesting from a theoretical point of view, it provides a model that
might be hard to implement as many aspects are hard to measure, e.g., the action of may have
interest in.</p>
      <p>In this work, we go towards a more functional model that can be implemented based on more
objective and easily accessible information. We aim at following the approach of network-based
models by adapting the framework to the e-participation context. The main difference with
previous works is that we aim at a model that is descriptive, based on observable data, and that
can be regularly implemented.</p>
    </sec>
    <sec id="sec-3">
      <title>3. Methodology</title>
      <sec id="sec-3-1">
        <title>3.1. Research questions</title>
        <p>The goal of this paper is to develop a model that describes the implementation phase of the
e-participation process. In order to build the proposed modelling approach we rely on Design
Research Science (DSR). This well-established research method allows creating an artifact, in
this case a model, by generating new knowledge for researchers but also directly usable by
practitioners, which are precisely the main goals of this work. DSR methodology can be defined
by three iterative cycles: the relevance cycle, the rigor cycle and the design cycle [21].</p>
        <p>As part of the rigor cycle, we initially reviewed the literature concerning the different online
and ofiflne participation methods and the existing models related to the description of these
methods. This review of the literature allows us to build upon existing models and to define which
are the important elements of the implementation of participation methods.</p>
        <p>
          To design and develop the proposed model, we make use of the existing theory in social
network analysis [
          <xref ref-type="bibr" rid="ref10">10</xref>
          ]. Based on the fundamental aspects of social network representation, we
define the key dimensions of participation processes extracted from the e-participation literature
so to properly generate all the elements of the network-based model.
        </p>
        <p>The relevance cycle in this work concerns the validation of the model in the citizen participation
context, and the possible added value provided by this new approach. These questions have
been initially answered by implementing the presented model to different use cases found in
the literature. This exercise helped to demonstrate that the model can be applied to different
methods, how it can be implemented, and to show its potential. This research design approach is
summarized in Fig. 1.</p>
        <sec id="sec-3-1-1">
          <title>Environment</title>
          <p>E-participation
methods
User cases
Relevance
cycle
Rigor
cycle</p>
        </sec>
        <sec id="sec-3-1-2">
          <title>Design</title>
          <p>Model
development
User cases
Design
cycle</p>
        </sec>
        <sec id="sec-3-1-3">
          <title>Knowledge base</title>
          <p>Participation</p>
          <p>models
Social Network</p>
          <p>Analysis</p>
          <p>The relevance cycle was carried out at the literature review level. We selected 4 use cases
where detailed information about their implementation could be found online, either in scientific
literature or on websites or and reports. The first two use cases were selected due to their
similarities in the goal of the e-participation, citizens involved in service development, and their
use of equivalent offline methods. The two other use cases showcase projects which were focused
on citizens involved in the decision making.</p>
        </sec>
      </sec>
    </sec>
    <sec id="sec-4">
      <title>4. Network-based model</title>
      <p>Citizen participation (consisting of online and ofiflne methods), as any collaborative process,
builds networks; networks of contacts, information and interactions [22]. Depending on these
interactions the outcome of the participation process might change. Equivalently, these factors
will affect the topology and structure of the network created [23].</p>
      <p>Participation methods imply and require social connections. These connections while not
representing all the aspects of the participation process, represent several important elements such
as: engagement, interactions and information. In this sense, tracking indicators associated to these
elements relying on a network-based model can become a very interesting tool for practitioners.</p>
      <p>
        The main focus during the implementation of this kind of models lies on the representation of
the different flows of information and connections between the agents involved. Following this
idea, we model participation processes as a social participation network [
        <xref ref-type="bibr" rid="ref10">10</xref>
        ].
      </p>
      <p>
        Network models are composed of two kinds of elements; nodes and edges. Nodes are the
agents composing the network, in this case the main elements of the participation process; actors,
tools or platforms. Edges represent the relations between agents, which in the participation
context describe the different kind of “ties” between these elements. In the following, these two
categories are further developed.
4.1. Nodes
During the participation process, different kinds of actors are involved. Common classifications
of the main e-participation actors include: citizens, policymakers, external stakeholders and
facilitators [
        <xref ref-type="bibr" rid="ref11">11</xref>
        ]. In the presented model, we group stakeholders based on their roles in the
participation process given that it will be their role which will define their importance and actions
in the participation method.
      </p>
      <p>We define as nodes of the network any kind of agent that generates, provides or collects
information. We initially define 4 types of nodes that represent different roles in the implementation
of participation methods:
• Participants: Citizens that are directly involved in one or multiple stages of the
participation method.
• Facilitators: Agents that collect information and coordinate the participation process.
• Target audience: Subset of citizens targeted by the participation process and the outcome
of the activity. Depending on the goal of the participation process, the target audience
might represent the whole population or only a specific part of it.
• Platform/tool: Elements of the participation method that receive, provide or analyze
information but are not active in nature. This is the case of prototypes or artifacts that are
generated as outcome of a given process.</p>
      <p>In some online participation methods, there might exist cyber equivalents of a human facilitator,
e.g, chatbots. In that case they are considered the same type of node as a human, as their function
is similar. On the other hand, artifacts that are built based on the requirements stated by the
participants or containers of information are considered as platforms and tools, given that they
lack of any coordination purpose.
4.2. Edges
The second element that constitutes a network model concerns the connections between the
different actors. These connections are represented by edges of the network.</p>
      <p>
        In social networks, there might exist different kinds of relations between the nodes that create
the graph [
        <xref ref-type="bibr" rid="ref10">10</xref>
        ]. The resulting networks can be modelled as multivariate graphs, where for a given
set of nodes there exist several sets of links describing different kinds of relationships [24]. In the
participation process, we defined the following types of edges:
• Communication: Communication is seen as any sharing of information, e.g. a discussion
between two actors or interaction between actors and platforms.
• Action: Interactions that are directly related to the creation of a service or another actor
actions. Usually, this is processing of information or the selection of proposals.
• Representation: Unilateral connection that defines how accurate are the attributes of the
targeted citizen represented by the associated participants.
      </p>
      <p>The several kinds of edges create a multivariate network that can be represented in one or
several figures [ 25]. These links are direct and the connection are not symmetric, such that
connections might have only one sense or have different strength depending on the sense of
the connection, creating a so-called direct graph. These edges might have different thickness
representing a stronger connection [16].</p>
    </sec>
    <sec id="sec-5">
      <title>5. Model implementation</title>
      <sec id="sec-5-1">
        <title>5.1. Implementation steps</title>
        <p>The goal of this paper is not only to present the approach and its application, but also to
provide enough information regarding its implementation. This implementation approach follows
traditional SNA methodology where first actors are identified, based on these actors, ties are
created, and then the network is visualized [26]. The proposed steps for the implementation of
the model are:
1. Define the nodes . Define which are the main actors and their roles in the participation
method. Start from the target audience identifying which are the group (or groups) that
the participation process is aiming at. Based on that target audience create the node(s)
of participants. Then, determine if there is any tool or platform used in this participation
phase, e.g. portal or survey platform. Finally, identify the facilitators, separating them if
there is any difference in their roles or activities.
2. Make the connections. The second step is to create the connections (communication,
action and representation) between the nodes. The main benefit of this step is to re-evaluate
the definition of the nodes, as the connections demand the understanding the role of each
group.
3. Visualization. The last step, visualization, helps to define the scope and the granularity of
the obtained model, as it will be displayed graphically.</p>
      </sec>
      <sec id="sec-5-2">
        <title>5.2. Visualization principles</title>
        <p>We define a systematic methodology to visualize and interpret the model outcome. To do so we
rely in common practices in heterogeneous multigraphs, where color codes are used as variables,
see [27, 28].</p>
        <p>• Nodes. Different colors represent different kind of nodes. In this case we propose; orange
as target audience, blue as participant, green as facilitator and red for the tools.
• Edges. Different kinds of line allow to differentiate the kind of connection between nodes.</p>
        <p>We propose to use; dotted lines as representation, solid lines as communication and dashed
lines for actions.
• Weights. Different line widths represent the different strength of the connections.</p>
        <p>Facilitator</p>
        <p>Processed
Practitioner data</p>
        <p>Tool
Digital tool</p>
        <p>Participants
participants
(group 1) RepresentTatiaonrget audience</p>
        <p>Citizenship</p>
        <p>Representation
participants
(group 2)</p>
        <p>Participants</p>
        <p>An example of the outcome of the model is depicted in Fig. 2. Note that this choice of colors
and line types is an initial attempt to obtain a consistent nomenclature.</p>
      </sec>
      <sec id="sec-5-3">
        <title>5.3. Use cases</title>
        <p>5.3.1. Use case 1: Social support e-service
This first use case was presented in [ 29]. It provides a scenario where an engineering requirements
approach is applied to create a social support e-Service in the Netherlands. The participation
process consists of: interviews with citizens and public servants, a low fidelity prototype and a
citizen walk-through. For the sake of comparison, we only focus on the interview’s stage during
our evaluation.</p>
        <p>In this case, the target audience is quite reduced; citizens with special needs and public servants
involved with the developed service. The participants are divided into citizens and public servants,
as they conduct two different kinds of interviews. Regarding the connections, we can identify the
representation relation between the target audience and the selected participants. In both cases
this connection is strong as there are two identified groups of participants that correspond to the
profiles of direct and indirect users. The method used are semi-structured interviews, so we can
establish a two-ways communication between interviews and interviewers. Then, based on the
information obtained, the practitioners create the user requirements which is considered as an
action. This provides the graph in Fig. 3a.
This use case focuses on the implementation of an e-government strategy at the city of La Louvière
(Belgium) thanks to the development of a Digital portal [30]. Three different participation methods
were applied; interviews and group discussions, prototyping and a survey. Again, we focus on the
interviews and group discussions, see Fig. 3b.</p>
        <p>Concerning the target audience, similarly to the previous case, the main targeted group were
citizens and public servants. Notably, in this case the citizens’ target audience was larger, as the
service was aimed at a broader group. In this implementation, there is no connection, i.e. lack
of participation, to one of the groups of target audience; citizens. Also, not all public servants
interviewed were actively involved in the creation of the Digital portal. This implies a lower
weight in the representation edges.</p>
        <p>As mentioned, when applying the model to this method implementation some potential issues
are easily spotted. For instance, the fact that there is a complete group of target audience that
is not connected to the process or that compared to the previous case the “representation" is
considered low for the case of public servants.</p>
        <p>Participants Target audience</p>
        <p>Citizens</p>
        <p>Citizens
Participants Target audience
sePruvbalnicts sePruvbalnicts</p>
        <p>Facilitator
5.3.3. Use case 3: Lets prepare Brussels
This use case showcases an initiative of the Brussels region (Belgium) to design new
environmental projects in the post-covid context1. In the first part of this participation process, a survey
was carried out by two entities; Dedicated company and Bruxelles Environnement. The survey
was performed in 3 different modalities; via an internet platform in the case of Bruxelles
Environnement, and internet or phone surveys in the case of Dedicated. The implementation of the
model can be seen in Fig. 4a.</p>
        <p>The participants are divided into 3 groups based on the method used for the interview. From
Fig. 4a it can be seen that the representation from citizens chosen for the survey platform is lower,
as the data indicated a clear bias in their selection, while the participants interviewed by telephone
have a stronger communication link as the average duration of their interaction (37 minutes) was
considerably longer than the ones by internet (18 minutes).
1https://letsprepare.monopinion.brussels</p>
        <p>This implementation of the model shows a more complex scenario with different instruments
used to collect data. In that regard, it provides good overview of how these methods are
implemented and where are the main differences in terms of application.
5.3.4. Use case 4: Healthy data project
This use case describes a public e-consultation related to the reuse of citizen health data2. The
consultation focused primarily on the countries of France, Belgium and the UK. A common
survey platform was used to gather all the contributions supported by a large-scale communication
campaign. The outcome of applying the proposed model to this implementation can be seen in
Fig. 4b.</p>
        <p>The target audience is divided into 3 groups based on the nationality. The representation edges
are only based on the number of participants of each country as there was no more information
due to the anonymity of the survey. Figure 4b shows the central importance of the survey platform
as it is the main interaction with participants and all data collection goes through this platform.</p>
        <p>Tool</p>
        <p>Survey platform
Facilitator</p>
        <p>Facilitator
Practitioners</p>
        <p>Practitioners</p>
        <p>Pdroactaessed
Report
Tool</p>
        <p>Participants
Citizens 3
(platform)
Participants
Citizens 2
(telephone)
Participants
Citizens 1
(internet)</p>
        <p>Target
Audience
Citizens</p>
        <p>Tool</p>
        <p>Survey platform
Facilitator
Practitioners</p>
        <p>Pdroactaessed
Report
Tool</p>
        <p>Participants</p>
        <p>Citizens
(France)
Participants</p>
        <p>Citizens
(UK)
Participants</p>
        <p>Citizens
(Belgium)</p>
      </sec>
    </sec>
    <sec id="sec-6">
      <title>6. Discussion</title>
      <sec id="sec-6-1">
        <title>6.1. Theoretical contributions</title>
        <p>The main theoretical contributions of this model to the research field of e-participation aim at
iflling the current gap in the description of how participation methods are implemented.</p>
        <p>
          The first contribution is related to the detailed characterization of online and offline methods
provided by the model. As already pointed by Macintosh [
          <xref ref-type="bibr" rid="ref11">11</xref>
          ], the proper comparison and
evaluation of different participation projects requires a more detailed characterization than the one
provided by the general categories commonly used. Compared to the characterization framework
proposed by [
          <xref ref-type="bibr" rid="ref11">11</xref>
          ], in this work we focus solely on the implementation phase. This approach allows
us to go a step further in the characterization of participation methods by providing an additional
layer of description, explicitly representing the interactions between actors and technology in the
form of a network. In that regard, we present a more functional perspective than other network
approaches such [20], as our model is only composed by observable and/or measurable attributes.
Thanks to that aspect, this model also fills the purpose and the need of having tools adapted for
less experienced and knowledgeable practitioners [
          <xref ref-type="bibr" rid="ref8 ref9">9, 8</xref>
          ].
        </p>
        <p>
          Additionally, this model helps to formalize the characterization of each method. This is in
line with current works aspiring at a formal definition of all e-participation elements, see [
          <xref ref-type="bibr" rid="ref5">5</xref>
          ].
This provides a standard methodology to represent both online and ofiflne methods, allowing a
practical evaluation of these different designs and boosting their future combination [
          <xref ref-type="bibr" rid="ref1 ref2">1, 2</xref>
          ].
        </p>
      </sec>
      <sec id="sec-6-2">
        <title>6.2. Implications for practitioners</title>
        <p>As mentioned, one of the main goals of this model is to become a useful tool for practitioners.
The use cases showcased in the previous section help to demonstrate this potential. For instance,
the need to define the different elements of the model, which demands an understanding and prior
analysis of the implementation setup, can be seen as an interesting asset. This is exemplified in
the definition of target audience nodes, which is clearly useful to identify which are the people to
be involved in the participation process.</p>
        <p>Another important aspect concerns the differentiation between communication and action
edges. In these cases, the difference between participants only communicating or taking active
actions might help to assess the level of participation achieved by the participation method [31].</p>
      </sec>
      <sec id="sec-6-3">
        <title>6.3. Limitations and future works</title>
        <p>This work is nonetheless an ongoing research subject to limitations. The model has been applied
to several examples found in the literature to test its reliability at a macroscopic level. However,
there is still need to further evaluate the additional utility of the model and to properly define
some of its elements. The latter issue refers to the fact that the current model still lacks precise
indicators concerning the edges and their weights associated.</p>
        <p>The evaluation of these aspects will be carried out in future works based on direct feedback
obtained from practitioners. In-depth interviews with practitioners will help to specifically identify
which are the main factors that should define the weights of the edges; e.g. communication or
representation. Then, practitioners will be asked to evaluate the model in terms of additional
utility, and to validate and rank the importance of the different elements depicted by the network
representation.</p>
      </sec>
    </sec>
    <sec id="sec-7">
      <title>7. Conclusions</title>
      <p>In this paper we introduced a network-based model to describe online and offline participation
methods. The goal was to facilitate the combination of different tools and methods by providing
a more detailed representation of all the elements involved in the implementation phase. This
model was built by using SNA theory so to represent the social interactions between actors, and
it was applied to several use cases to show its reliability and potential application. This paper
represents a first step in the development of a network-based model and, as such, it focused on
the definition and introduction of the main concepts.</p>
      <p>The main contributions of the presented model are manifold: i) it focuses on a single phase of
the e-participation process allowing more granularity in its description, ii) it allows the study of
the complementary between between online and ofiflne methods, and the comparison between
applications, iii) it can be used during the planning phase once the policy and target audience are
defined or during the evaluation process, and iv) it can be easily implemented based on retrievable
information.
[15] C. Atkinson, M. Gutheil, K. Kiko, On the relationship of ontologies and models, GI-Edition:</p>
      <p>Lecture Notes in Informatics. Proceedings 96 (2006) 47–60.
[16] K. Kautz, Applying Complex Adaptive Systems Theory to Understand Distributed
Participatory Design in Contemporary, Crowdsourced Information Systems Development,
Australasian Journal of Information Systems 24 (2020).
[17] M. Ignaccolo, G. Inturri, C. Garofalo, A. Pluchino, A. Rapisarda, Agent-based Modelling
of Stakeholder Interaction in Transport Decisions (2013).
[18] K. M. Abbasi, T. A. Khan, I. u. Haq, Framework for Integrated Use of Agent-Based and</p>
      <p>Ambient-Oriented Modeling, Mathematics 10 (2022) 4157.
[19] L. Dai, Q. Han, B. de Vries, Agent-Based Simulation of Citizen Participation in
Nature</p>
      <p>Based Projects, SSRN Electronic Journal (2022).
[20] G. Piperagkas, R. Angarita, V. Issarny, Social Participation Network: Linking Things,
Services and People to Support Participatory Processes, in: Advanced Information Systems
Engineering Workshops, volume 382, Springer International Publishing, Cham, 2020, pp.
109–120.
[21] A. Hevner, A. R, S. March, S. T, Park, J. Park, Ram, Sudha, Design Science in Information</p>
      <p>Systems Research, Management Information Systems Quarterly 28 (2004) 75–.
[22] J. E. Innes, D. E. Booher, Reframing public participation: strategies for the 21st century,</p>
      <p>Planning Theory &amp; Practice 5 (2004) 419–436.
[23] I. Amarasinghe, S. Manske, H. U. Hoppe, P. Santos, D. Hernández-Leo, Using Network
Analysis to Characterize Participation and Interaction in a Citizen Science Online
Community, in: Collaboration Technologies and Social Computing, volume 12856, Cham, 2021,
pp. 67–82.
[24] C. Muelder, L. Gou, K.-L. Ma, M. X. Zhou, Multivariate Social Network Visual Analytics,
in: Multivariate Network Visualization, volume 8380, Springer International Publishing,
Cham, 2014, pp. 37–59.
[25] S. Wang, S. Chen, T. Cai, Y. Wu, MULTI-NETVIS: Visual Analytics for Multivariate</p>
      <p>Network, Applied Sciences 12 (2022) 8405.
[26] L. Shi, Q. Liao, H. Tong, Y. Hu, C. Wang, C. Lin, W. Qian, OnionGraph: Hierarchical
topology+attribute multivariate network visualization, Visual Informatics 4 (2020) 43–57.
[27] H. Hu, L. Wu, C. Yang, H. Song, Interactive multigraph visualization and exploration with
a two-phase strategy, Journal of Systems Engineering and Electronics 25 (2014) 886–894.
[28] D. Redondo, A. Sallaberry, D. Ienco, F. Zaidi, P. Poncelet, Layer-Centered Approach
for Multigraphs Visualization, in: 2015 19th International Conference on Information
Visualisation, IEEE, Barcelona, Spain, 2015, pp. 50–55.
[29] L. van Velsen, T. van der Geest, M. ter Hedde, W. Derks, Requirements engineering for
e-Government services: A citizen-centric approach and case study, Government Information
Quarterly 26 (2009) 477–486.
[30] A. Simonofski, B. Vanderose, A. Clarinval, M. Snoeck, The Impact of User Participation
Methods on E-Government Projects: The Case of La Louvière, Belgium, Media and
Communication 6 (2018) 175–186.
[31] N. Lago, M. Durieux, C. Scoubeau, C. Elsen, C. Schelings, Citizen Participation through
Digital Platforms: the Challenging Question of Data Processing for Cities (2019).</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          [1]
          <string-name>
            <given-names>V.</given-names>
            <surname>Pina</surname>
          </string-name>
          ,
          <string-name>
            <given-names>L.</given-names>
            <surname>Torres</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Royo</surname>
          </string-name>
          ,
          <article-title>Comparing online with offline citizen engagement for climate change: Findings from Austria, Germany</article-title>
          and Spain,
          <source>Government Information Quarterly</source>
          <volume>34</volume>
          (
          <year>2017</year>
          )
          <fpage>26</fpage>
          -
          <lpage>36</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          [2]
          <string-name>
            <surname>K.-T. Tai</surname>
            , G. Porumbescu,
            <given-names>J.</given-names>
          </string-name>
          <string-name>
            <surname>Shon</surname>
          </string-name>
          ,
          <article-title>Can e-participation stimulate offline citizen participation: an empirical test with practical implications</article-title>
          ,
          <source>Public Management Review</source>
          <volume>22</volume>
          (
          <year>2020</year>
          )
          <fpage>278</fpage>
          -
          <lpage>296</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          [3]
          <string-name>
            <given-names>A.</given-names>
            <surname>Santamaria-Philco</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J. H.</given-names>
            <surname>Canos Cerda</surname>
          </string-name>
          , M. C.
          <article-title>Penades Gramaje, Advances in eParticipation: A perspective of Last Years, IEEE Access 7 (</article-title>
          <year>2019</year>
          )
          <fpage>155894</fpage>
          -
          <lpage>155916</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          [4]
          <string-name>
            <given-names>S.</given-names>
            <surname>Scherer</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M. A.</given-names>
            <surname>Wimmer</surname>
          </string-name>
          ,
          <article-title>A Metamodel for the E-Participation Reference Framework</article-title>
          , in: Electronic Participation, volume
          <volume>9821</volume>
          , Springer International Publishing, Cham,
          <year>2016</year>
          , pp.
          <fpage>3</fpage>
          -
          <lpage>16</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          [5]
          <string-name>
            <given-names>L.</given-names>
            <surname>Porwol</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Ojo</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J. G.</given-names>
            <surname>Breslin</surname>
          </string-name>
          ,
          <article-title>An ontology for next generation e-Participation initiatives</article-title>
          ,
          <source>Government Information Quarterly</source>
          <volume>33</volume>
          (
          <year>2016</year>
          )
          <fpage>583</fpage>
          -
          <lpage>594</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          [6]
          <string-name>
            <given-names>M.</given-names>
            <surname>Toots</surname>
          </string-name>
          ,
          <article-title>Why E-participation systems fail: The case of Estonia's Osale</article-title>
          .ee,
          <source>Government Information Quarterly</source>
          <volume>36</volume>
          (
          <year>2019</year>
          )
          <fpage>546</fpage>
          -
          <lpage>559</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          [7]
          <string-name>
            <given-names>M.</given-names>
            <surname>Steinbach</surname>
          </string-name>
          ,
          <string-name>
            <given-names>N.</given-names>
            <surname>Wilker</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Schöttle</surname>
          </string-name>
          ,
          <article-title>E-participation on the local level - A census survey approach for researching its implementation</article-title>
          ,
          <source>Journal of Information Technology &amp; Politics</source>
          <volume>17</volume>
          (
          <year>2020</year>
          )
          <fpage>12</fpage>
          -
          <lpage>32</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          [8]
          <string-name>
            <given-names>C. S.</given-names>
            <surname>Slotterback</surname>
          </string-name>
          ,
          <source>Planners' Perspectives on Using Technology in Participatory Processes, Environment and Planning B: Planning and Design</source>
          <volume>38</volume>
          (
          <year>2011</year>
          )
          <fpage>468</fpage>
          -
          <lpage>485</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          [9]
          <string-name>
            <given-names>S.</given-names>
            <surname>Shahab</surname>
          </string-name>
          ,
          <string-name>
            <given-names>B.</given-names>
            <surname>Bagheri</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R.</given-names>
            <surname>Potts</surname>
          </string-name>
          ,
          <article-title>Barriers to employing e-participation in the Iranian planning system</article-title>
          ,
          <source>Cities</source>
          <volume>116</volume>
          (
          <year>2021</year>
          )
          <fpage>103281</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          [10]
          <string-name>
            <given-names>S.</given-names>
            <surname>Wasserman</surname>
          </string-name>
          ,
          <string-name>
            <given-names>K.</given-names>
            <surname>Faust</surname>
          </string-name>
          ,
          <article-title>Social network analysis: Methods and applications (</article-title>
          <year>1994</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          [11]
          <string-name>
            <given-names>A.</given-names>
            <surname>Macintosh</surname>
          </string-name>
          ,
          <article-title>Characterizing e-participation in policy-making</article-title>
          ,
          <source>in: 37th Annual Hawaii International Conference on System Sciences</source>
          ,
          <year>2004</year>
          . Proceedings of the, IEEE,
          <string-name>
            <surname>Big</surname>
            <given-names>Island</given-names>
          </string-name>
          ,
          <string-name>
            <surname>HI</surname>
          </string-name>
          , USA,
          <year>2004</year>
          , p.
          <volume>10</volume>
          pp.
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          [12]
          <string-name>
            <surname>A. De Nicola</surname>
            ,
            <given-names>M. L.</given-names>
          </string-name>
          <string-name>
            <surname>Villani</surname>
            , Smart City Ontologies and
            <given-names>Their</given-names>
          </string-name>
          <string-name>
            <surname>Applications</surname>
            :
            <given-names>A Systematic</given-names>
          </string-name>
          <string-name>
            <surname>Literature</surname>
            <given-names>Review</given-names>
          </string-name>
          ,
          <source>Sustainability</source>
          <volume>13</volume>
          (
          <year>2021</year>
          )
          <fpage>5578</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          [13]
          <string-name>
            <given-names>M.</given-names>
            <surname>Yusuf</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C.</given-names>
            <surname>Adams</surname>
          </string-name>
          ,
          <string-name>
            <given-names>K.</given-names>
            <surname>Dingley</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A Novel</given-names>
            <surname>Framework of</surname>
          </string-name>
          e-Participation,
          <year>2014</year>
          , p.
          <fpage>363</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          [14]
          <string-name>
            <given-names>M. S.</given-names>
            <surname>Islam</surname>
          </string-name>
          ,
          <article-title>Towards a sustainable e-Participation implementation model</article-title>
          ,
          <source>European journal of ePractice 5</source>
          (
          <year>2008</year>
          ).
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>