<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>Participation in a New Media Landscape - a Literature Review of Participation Tools</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Marius Rohde Johannessen</string-name>
          <email>marius.johannessen@usn.no</email>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Eiri Elvestad</string-name>
          <email>eiri.elvestad@media.uio.no</email>
        </contrib>
      </contrib-group>
      <fpage>27</fpage>
      <lpage>35</lpage>
      <abstract>
        <p>In this paper, we present our preliminary findings from a literature analysis of eParticipation tools. The background for the study is the expectation from many national governments that local politicians should have an ongoing digital dialogue with citizens. Media technologies and communication tools frame, structure and shape debates based on their affordances, underlying sociocultural as well as sociotechnical context. The purpose of this work in progress is to present our initial findings from a literature study of eParticipation tools. The literature study will form the foundation for future work, which includes a further examination of the potential political/deliberative consequences of different technologies. The study is set in a Norwegian context.</p>
      </abstract>
      <kwd-group>
        <kwd>eParticipation</kwd>
        <kwd>democracy tools</kwd>
        <kwd>literature review</kwd>
        <kwd>deliberation</kwd>
        <kwd>political communication</kwd>
      </kwd-group>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>1. Introduction and Background</title>
      <p>of the constitution, §100: "Government is required to facilitate an open, inclusive and enlightened
public debate" (authors' translation).</p>
      <p>
        Despite this objective, democratic interest and participation is declining in many areas. In 2017, 7
percent of Norwegians were members of a political party and only 2 percent considered themselves
as active members (attending meetings etc.)
        <xref ref-type="bibr" rid="ref20">(SSB, 2017)</xref>
        . At the same time, our options for
participating have never been better. Social media, for example, allow us to express our political
opionions and take part in public debate, and is also an important tool for mobilization and protest
        <xref ref-type="bibr" rid="ref21">(Steen-Johnsen, Enjolras &amp; Wollebaek, 2013)</xref>
        .
1.1
      </p>
      <sec id="sec-1-1">
        <title>Local Political Debates and Participation in a New Media Landscape</title>
        <p>
          The Internet brought about a belief that more people would participate in democratic discourse,
both nationally and locally. Research on the democratic effects of digital tools have to some extent
dampened this belief. New media and participation channels introduce new dilemmas for
participation, and the Internet has not necessarily led to new groups of people being engaged, or a
more deliberative public debate
          <xref ref-type="bibr" rid="ref19">(Skogerbø &amp; Enli, 2008)</xref>
          .
        </p>
        <p>
          Early attempts at democratic discussion forums showed that few were able to attract participants,
and those who did participate often disappeared after an initial burst of interest (Saebø, Rose, &amp;
Nyvang, 2009). When social media appeared, there was much interest from local politicians,
municipalities, and political parties. In social media such as Twitter, blogs and Facebook, active
citizens formed protest groups and discussed politics, and local politicians attempted to tap into this
engagement
          <xref ref-type="bibr" rid="ref3">(Brandtzaeg &amp; Lüders, 2008)</xref>
          . Based on the idea that participation has not disappeared,
but rather found new forms, many municipalities established a presence in social media
          <xref ref-type="bibr" rid="ref17">(Segerberg
&amp; Bennett, 2011)</xref>
          . While social media has made it easier to engage in political debate, the quality of
debates has not necessarily benefited
          <xref ref-type="bibr" rid="ref10">(Johannessen, 2018)</xref>
          . Debates are often fragmented and with
little direction, which makes it difficult to include in formal political processes
          <xref ref-type="bibr" rid="ref13">(Majumdar, 2017)</xref>
          .
Policy informatics emerged as an approach to collect and analyze unstructured social media data
through data mining and machine-learning
          <xref ref-type="bibr" rid="ref1">(Androutsopoulou, Mureddu, Loukis, &amp; Charalabidis,
2016)</xref>
          .
        </p>
        <p>
          In Norway, the organization for municipal cooperation (KS) has published two studies showing
that on the local level, municipalities have been successful in using social media for information,
reputation building and crisis management, but less successful in tapping into the deliberative
possibilities of social media12. While politicians on the national level have become active social media
users, local politicians have been more reluctant, with a few being very active but most being absent
          <xref ref-type="bibr" rid="ref10">(Enli &amp; Skogerbø, 2013; Elvestad &amp; Johannessen, 2017)</xref>
          .
        </p>
        <p>1
http://www.ks.no/globalassets/vedlegg-til-hvert-fagomrader/utvikling/fou/politisk-lederskap-ogdialog-gjennom-sosiale-medier.pdf</p>
        <p>2
http://www.ks.no/contentassets/40a67437a2b3485d9e995fe061fee0f0/kommunesektorens-bruk-avsosiale-medier.pdf</p>
        <p>
          A third option, tools tailor-made for participation, was emphasized in the EU FP7 funding
programme, and from around 2013/2014 we started to see new participation tools presented at
conferences (see f.ex.
          <xref ref-type="bibr" rid="ref11">Liddo &amp; Shum, 2014</xref>
          ; Porwol, Ojo, &amp; Breslin, 2014
        </p>
        <p>So far, there has been little research into how different tailor-made tools or participation can be
applied in Norwegian local communities. The purpose of our project is therefore to summarize and
map the different tools presented by literature (this work in progress-paper), and further to examine
if and how these tools are being used, and how they can be used to further participation. This
introduces the following research questions:</p>
        <sec id="sec-1-1-1">
          <title>RQ1: What can we learn from exisiting research on tools for deliberation?</title>
        </sec>
        <sec id="sec-1-1-2">
          <title>RQ2: How can we categorize and sort different types of tools and their usage areas?</title>
        </sec>
        <sec id="sec-1-1-3">
          <title>RQ3: How can we measure the usefulness of such tools for (Norwegian) local democracy?</title>
        </sec>
      </sec>
    </sec>
    <sec id="sec-2">
      <title>2. Research Approach</title>
      <p>
        In this paper, we mainly address RQ1, and present our preliminary findings from a literature review
of participation tools. Literature reviews are important tools for summarizing research in a given
field or subject, as long as the review clearly states how data collection and analysis was done so
others can replicate the review
        <xref ref-type="bibr" rid="ref22">(Templier &amp; Pare, 2015)</xref>
        . Our process was as follows:
      </p>
      <p>We downloaded the Digital government research library V16.63 (DGRL), as DGRL covers most
of the relevant journals and conferences for e-Government and e-Democracy. We used Endnote to
search the DGRL (title, abstract and keyword sections) for variations and combinations of the
following keywords:
tools, (e-)democracy, (e-)participation, deliberation, consultation, policy modelling
To be included in the list, articles should describe a specific tool or method for democratic
participation. This narrowed down the list quite a lot, as few articles address concrete tools. Further,
we classified the relevant articles according to the following:</p>
      <p>Type of system (what kind of participation, thematic area)
Research focus (evaluation/implementation, development)
Empirically validated (yes, no)</p>
      <p>To narrow our selection, we first selected based on article title and keywords, and further
narrowed the list by reading the abstracts to search for specific democracy tools.</p>
      <p>The DGRL contains 14.940 articles published between 1995 and 2020. More than 1300 of these
address digital (e-) democracy, around 600 address social media, 154 social media + deliberation or
3 http://faculty.washington.edu/jscholl/category/egrl/
consultation and 158 contains the combination "tools" and "democracy". Finally, we searched for
open data + deliberation or consultation, which provided 39 hits in the DGRL.</p>
      <p>After the second screening, we were left with 49 articles (see table 1 below for the resulting
literature matrix). Many of the articles in the initial sorting discussed participation in general terms
with no mentions of specific tools, or social media without specific methods or frameworks for data
use. These were not included, as we concentrated on articles explicitly discussing development or
evaluation of specific tools. If we had included case studies of participation in geographic regions or
social media in general, we would have been left with a much larger list. The articles identified in
the DGRL mainly divided into two categories: Development and evaluation. Most of the articles
have empirical data, mostly case studies or evalations of pilot projects. Finally, it is worth noting
that there are a lot more democracy tools than studies of democracy tools. Many of these are
presented in the Oxford Internet Institutes «Civic tech field guide» 4, but as our focus was academic
research, we have not included these in the current phase of our project. We mention the Oxford
guide simply to acknowledge that there are a lot of tools out there, and a great need for research into
their implementation, use and effects.</p>
    </sec>
    <sec id="sec-3">
      <title>3. Preliminary Findings</title>
      <p>The first thing we looked for was what type of democratic purpose or democratic model the articles
discussed, and there is great variation in how explicit the included articles are in this regard. Some
mention a democracy model or purpose, but most leave this up to the reader. The generic term
"participation technology" is the most commonly used. The articles are more focused on concrete
usage areas, and in our DGRL sample we found four categories that broadly fit all the articles:
Participatory budgeting, urban planning (including smart cities), open data and policy informatics.
In addition, we found the categories deliberation and consultation implicitly mentioned in many
articles.</p>
      <p>4
https://docs.google.com/spreadsheets/d/1FzmvVAKOOFdixCs7oz88cz9g1fFPHDlg0AHgHCwhf4A/html
view#</p>
      <p>
        Five articles discuss participatory budgeting (PB). PB in short allows citizens to "make their own"
budget. In its simplest form it is about distributing funds to various areas, so people can see the
consequences of distributing limited funds. More money to health means less for education or
transport. More advanced PB solutions also simulates citizen sentiment towards the budget, for
example reactions to a tax hike. In other scenarios, PB sets aside a sum of money for actual citizen
distribution
        <xref ref-type="bibr" rid="ref18">(Sintomer, 2008)</xref>
        . In our sample, we find evalution of concrete cases
        <xref ref-type="bibr" rid="ref23">(Zepic, Dapp, &amp;
Krcmar, 2017)</xref>
        and descriptions of tools or designs for PB
        <xref ref-type="bibr" rid="ref14">(Omar, Weerakkody, &amp; Sivarajah, 2017)</xref>
        .
Tool-oriented articles tend to focus more on how the systems works, and not as much on democratic
effects, outcomes or contextual issues.
      </p>
      <p>
        The second category is urban planning, with 12 articles. In this category we find articles
presenting designs for systems, or descriptions of systems aimed at including citizens in various
stages of urban planning. One is an evaluation of the City of Chicago's tool for dialogue, a system
actually in use,
        <xref ref-type="bibr" rid="ref12">(Lyons, Walsh, Aleman, &amp; Robinson, 2014)</xref>
        , while most of the other articles discuss
development of systems, some in the design stage, while others have been run as pilot projects.
      </p>
      <p>
        Open data also receives some attention in a democratic setting. Open data is publicly available
data sets, published in a machine-readable form under an open data license which allows users to
use, reuse and redistribute the data. An important motivation for open data is to provide citizens
with information which allows them to reach reasoned conclusions and opinions when participating
in democratic activities
        <xref ref-type="bibr" rid="ref2">(Attard, 2015)</xref>
        . The articles in our sample discuss how open data can be
essential for deliberation and transparency, but also point out that there is little research on how to
realize the democratic potential of open data
        <xref ref-type="bibr" rid="ref8">(Hansson, Belkacem, &amp; Ekenberg, 2015)</xref>
        . Other articles
are critical towards the statement that simply publishing data will lead to increased engagement
        <xref ref-type="bibr" rid="ref9">(Hellberg, 2014)</xref>
        , or point out that even if open data has democratic potential, the learning curve is
steep and there are few good, non-technical tools out there to process data sets
        <xref ref-type="bibr" rid="ref7">(Graves &amp; Hendler,
2013)</xref>
        . This category is somewhat special, as it does not explicitly mention concrete tools. However,
we mention it because there has been much debate on open data as an underlying requirement of a
reasoned debate. As such, it is heavily intertwined with the policy informatics/analytics tools we
found.
      </p>
      <p>
        The fourth category we identified involves using and analyzing data, which is not necessarily
part of a citizen dialogue but rather the result of citizens actions or statements. Policy informatics
(and/or data analytics) concerns the use the use of analytical tools for unstructured data from
sources such as social media. Many of these articles mention that rather than trying to establish new
arenas or tools, we should concentrate on what is there, and use algorithms and analytical
techniques to make sense of the discussion that is already taking place in social media and other
digital platforms. This can for example be used as an advanced form of polling and included in the
political decision-making process. One of the articles take a rather elitist approach and point out that
today political decision-making has become such a complex and intertwined area that what we
really need is not another platform for debate, but access to experts, and suggests a system for
identifying experts in varying fields
        <xref ref-type="bibr" rid="ref1">(Androutsopoulou et al., 2016)</xref>
        .
      </p>
      <p>The next categories we identified are deliberation and consultation. In this area there is a variety
of articles discussing concrete cases, tools and systems for consultation, and quite a few presenting
a more generic approach or framework for deliberation. The latter was excluded from our screening,
but we still retained a few articles just to present an overview of what is being discussed in relation
to deliberative systems. More work is needed to examine the papers in this category more closely,
but so far we see that few articles are explicit in stating what kind of dialogue they want to facilitate,
who initiates it, the impact it should have in the political decision-making process, or where in the
process results for deliberation should be applied. The evaluation articles in this category indicates
that open discussions provide little in terms of constructive participation, but systems that require
more preparation and knowledge from the user can provide more valuable input. The catch is that
these complex systems also struggle with few users.</p>
    </sec>
    <sec id="sec-4">
      <title>4. Discussion and Future Research</title>
      <p>Overall, the literature on democracy tools tend to present results from limited pilot studies, present
results in terms of quantitative figures (with some discussion on qualitative sucess criteria). We have
identified a potential catch-22 in that systems with high quality feedback tend to attract few users,
while systems with many users tend to generate low quality. This is by no means certain, but needs
further research. Finally, we see that most tool-based articles focus on the technology, with little or
no discussion on the democractic context/model or outcomes, which makes theorizing on
tool/democracy combinations another avenue for further research. Future plans include the
abovementioned, as well as more empirical work on how Norwegian municipalities use (or can use) these
tools to further local democratic participation.</p>
      <sec id="sec-4-1">
        <title>About the Authors</title>
        <p>Marius Rohde Johannessen
Marius Rohde Johannessen is Associate Professor in information systems at the University of South-Eastern
Norway. His research interests include communication, digitalisation in the public sector, especially digital
democracy and political communication, as well as smart cities. He is currently involved in the GoForIT
network for sustainability in IT education.</p>
        <p>Eiri Elvestad</p>
      </sec>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          <string-name>
            <surname>Androutsopoulou</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Mureddu</surname>
            ,
            <given-names>F.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Loukis</surname>
            ,
            <given-names>E.</given-names>
          </string-name>
          , &amp;
          <string-name>
            <surname>Charalabidis</surname>
            ,
            <given-names>Y.</given-names>
          </string-name>
          (
          <year>2016</year>
          ).
          <article-title>Passive Expert-Sourcing for Policy Making in the European Union</article-title>
          . In E. Tambouris, P. Panagiotopoulos, Ø. Saebø,
          <string-name>
            <given-names>M. A.</given-names>
            <surname>Wimmer</surname>
          </string-name>
          ,
          <string-name>
            <given-names>T. A.</given-names>
            <surname>Pardo</surname>
          </string-name>
          ,
          <string-name>
            <given-names>Y.</given-names>
            <surname>Charalabidis</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D. Sa</given-names>
            <surname>Soares</surname>
          </string-name>
          , &amp; T. Janowski (Eds.),
          <source>Electronic Participation: Proceedings of the 8th IFIP WG 8</source>
          .5 International Conference, ePart
          <year>2016</year>
          (pp.
          <fpage>162</fpage>
          -
          <lpage>175</lpage>
          ). Guimarães, Portugal: Springer International Publishing.
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          <string-name>
            <surname>Attard</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Orlandi</surname>
            ,
            <given-names>F.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Scerri</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          , &amp;
          <string-name>
            <surname>Auer</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          (
          <year>2015</year>
          ).
          <article-title>A systematic review of open government data initiatives</article-title>
          .
          <source>Government Information Quarterly</source>
          ,
          <volume>32</volume>
          (
          <issue>4</issue>
          ),
          <fpage>399</fpage>
          -
          <lpage>418</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          <string-name>
            <surname>Brandtzaeg</surname>
            ,
            <given-names>P. B.</given-names>
          </string-name>
          , &amp;
          <string-name>
            <surname>Lüders</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          (
          <year>2008</year>
          ).
          <source>eCitizen 2</source>
          .
          <article-title>0: The Ordinary Citizen as a Supplier of Public Sector Information</article-title>
          . Retrieved from Oslo:
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          <string-name>
            <surname>Brooks</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          , &amp;
          <string-name>
            <surname>Manza</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          (
          <year>2007</year>
          ).
          <article-title>Why welfare states persist: The importance of public opinion in democracies</article-title>
          . Chicago: University of Chigaco Press.
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          <string-name>
            <surname>Dewey</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          (
          <year>1927</year>
          ).
          <source>The Public and its Problem</source>
          . Athens: Swallow Press/Ohio University Press.
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          <string-name>
            <surname>Ferree</surname>
            ,
            <given-names>M. M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Gamson</surname>
            ,
            <given-names>W. A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Gerhards</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          , &amp;
          <string-name>
            <surname>Rucht</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          (
          <year>2002</year>
          ).
          <article-title>Four models of the public sphere in modern democracies</article-title>
          .
          <source>Theory and society</source>
          ,
          <volume>31</volume>
          (
          <issue>3</issue>
          ),
          <fpage>289</fpage>
          -
          <lpage>324</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          <string-name>
            <surname>Graves</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          , &amp;
          <string-name>
            <surname>Hendler</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          (
          <year>2013</year>
          ).
          <source>Visualization Tools for Open Government Data Proceedings of the 14th Annual International Conference on Digital Government Research</source>
          (pp.
          <fpage>136</fpage>
          -
          <lpage>145</lpage>
          ). New York, NY, USA: ACM.
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          <string-name>
            <surname>Hansson</surname>
            ,
            <given-names>K.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Belkacem</surname>
            ,
            <given-names>K.</given-names>
          </string-name>
          , &amp;
          <string-name>
            <surname>Ekenberg</surname>
            ,
            <given-names>L.</given-names>
          </string-name>
          (
          <year>2015</year>
          ).
          <article-title>Open Government and Democracy: A Research Review</article-title>
          . Social Science Computer Review,
          <volume>33</volume>
          (
          <issue>5</issue>
          ),
          <fpage>540</fpage>
          -
          <lpage>555</lpage>
          . doi:
          <volume>10</volume>
          .1177/0894439314560847
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          <string-name>
            <surname>Hellberg</surname>
            ,
            <given-names>A.-S.</given-names>
          </string-name>
          (
          <year>2014</year>
          ). Policy, Process, People and
          <string-name>
            <given-names>Public</given-names>
            <surname>Data</surname>
          </string-name>
          . In M. Janssen,
          <string-name>
            <given-names>H. J.</given-names>
            <surname>Scholl</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M. A.</given-names>
            <surname>Wimmer</surname>
          </string-name>
          , &amp; F. Bannister (Eds.),
          <source>Electronic Government: Proceedings of the 13th IFIP WG 8</source>
          .5 International Conference,
          <source>EGOV 2014</source>
          (Vol.
          <volume>8653</volume>
          , pp.
          <fpage>265</fpage>
          -
          <lpage>276</lpage>
          ). Dublin, Ireland.
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          <string-name>
            <surname>Johannessen</surname>
            ,
            <given-names>M. R.</given-names>
          </string-name>
          (
          <year>2018</year>
          ).
          <article-title>Genres of Participation in Social Networking Systems: A Study of the 2017 Norwegian Parliamentary Election</article-title>
          , Cham.
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          <string-name>
            <surname>Liddo</surname>
            ,
            <given-names>A. D.</given-names>
          </string-name>
          , &amp;
          <string-name>
            <surname>Shum</surname>
            ,
            <given-names>S. B.</given-names>
          </string-name>
          (
          <year>2014</year>
          ).
          <article-title>New Ways of Deliberating Online: An Empirical Comparison of Network and Threaded Interfaces for Online Discussion</article-title>
          . In E. Tambouris,
          <string-name>
            <given-names>A.</given-names>
            <surname>Macintosh</surname>
          </string-name>
          , &amp; F. Bannister (Eds.),
          <source>Electronic Participation: Proceedings of the 6th IFIP WG 8</source>
          .5 International Conference, ePart
          <year>2014</year>
          (Vol.
          <volume>8654</volume>
          , pp.
          <fpage>90</fpage>
          -
          <lpage>101</lpage>
          ). Dublin, Ireland.
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          <string-name>
            <surname>Lyons</surname>
            ,
            <given-names>S. H.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Walsh</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Aleman</surname>
            ,
            <given-names>E.</given-names>
          </string-name>
          , &amp;
          <string-name>
            <surname>Robinson</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          (
          <year>2014</year>
          ).
          <article-title>Exploring regional futures: Lessons from Metropolitan Chicago's online MetroQuest</article-title>
          .
          <source>Technological Forecasting and Social Change</source>
          ,
          <volume>82</volume>
          ,
          <fpage>23</fpage>
          -
          <lpage>33</lpage>
          . doi:
          <volume>10</volume>
          .1016/j.techfore.
          <year>2013</year>
          .
          <volume>05</volume>
          .009
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          <string-name>
            <surname>Majumdar</surname>
            ,
            <given-names>S. R.</given-names>
          </string-name>
          (
          <year>2017</year>
          ).
          <article-title>The case of public involvement in transportation planning using social media</article-title>
          .
          <source>Case Studies on Transport Policy</source>
          ,
          <volume>5</volume>
          (
          <issue>1</issue>
          ),
          <fpage>121</fpage>
          -
          <lpage>133</lpage>
          . doi:
          <volume>10</volume>
          .1016/j.cstp.
          <year>2016</year>
          .
          <volume>11</volume>
          .002
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          <string-name>
            <surname>Omar</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Weerakkody</surname>
            ,
            <given-names>V.</given-names>
          </string-name>
          , &amp;
          <string-name>
            <surname>Sivarajah</surname>
            ,
            <given-names>U.</given-names>
          </string-name>
          (
          <year>2017</year>
          ).
          <article-title>Developing Criteria for Evaluating a Multi-channel Digitally Enabled Participatory Budgeting Platform 9th International Conference on Electronic Participation (ePart</article-title>
          <year>2017</year>
          ) (pp.
          <fpage>3</fpage>
          -
          <lpage>11</lpage>
          ). St. Petersburg, Russia: Springer, Cham.
        </mixed-citation>
      </ref>
      <ref id="ref15">
        <mixed-citation>
          <string-name>
            <surname>Oppenheim</surname>
            ,
            <given-names>F. E.</given-names>
          </string-name>
          (
          <year>1971</year>
          ).
          <article-title>DEMOCRACY - CHARACTERISTICS INCLUDED AND EXCLUDED</article-title>
          . The Monist,
          <volume>55</volume>
          (
          <issue>1</issue>
          ),
          <fpage>29</fpage>
          -
          <lpage>50</lpage>
          . doi:
          <volume>10</volume>
          .2307/27902204
        </mixed-citation>
      </ref>
      <ref id="ref16">
        <mixed-citation>
          <string-name>
            <surname>Porwol</surname>
            ,
            <given-names>L.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Ojo</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          , &amp;
          <string-name>
            <surname>Breslin</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          (
          <year>2014</year>
          ).
          <article-title>A Semantic Deliberation Model for e-Participation</article-title>
          . In M. Janssen,
          <string-name>
            <given-names>F.</given-names>
            <surname>Bannister</surname>
          </string-name>
          ,
          <string-name>
            <given-names>O.</given-names>
            <surname>Glassey</surname>
          </string-name>
          ,
          <string-name>
            <given-names>H. J.</given-names>
            <surname>Scholl</surname>
          </string-name>
          , E. Tambouris,
          <string-name>
            <given-names>M. A.</given-names>
            <surname>Wimmer</surname>
          </string-name>
          , &amp; A.
          <string-name>
            <surname>Macintosh</surname>
          </string-name>
          (Eds.),
          <source>Electronic Government and Electronic Participation: Joint Proceedings of Ongoing Research and Projects of IFIP WG 8.5 EGOV and ePart 2014</source>
          (Vol.
          <volume>21</volume>
          , pp.
          <fpage>40</fpage>
          -
          <lpage>50</lpage>
          ). Dublin, Ireland.
        </mixed-citation>
      </ref>
      <ref id="ref17">
        <mixed-citation>
          <string-name>
            <surname>Segerberg</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          , &amp;
          <string-name>
            <surname>Bennett</surname>
            ,
            <given-names>W. L.</given-names>
          </string-name>
          (
          <year>2011</year>
          ).
          <article-title>Social Media and the Organization of Collective Action: Using Twitter to Explore the Ecologies of Two Climate Change Protests</article-title>
          .
          <source>The Communication Review</source>
          ,
          <volume>14</volume>
          (
          <issue>3</issue>
          ),
          <fpage>197</fpage>
          -
          <lpage>215</lpage>
          . doi:
          <volume>10</volume>
          .1080/10714421.
          <year>2011</year>
          .597250
        </mixed-citation>
      </ref>
      <ref id="ref18">
        <mixed-citation>
          <string-name>
            <surname>Sintomer</surname>
            ,
            <given-names>Y.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Herzberg</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          , &amp;
          <string-name>
            <surname>Röcke</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          (
          <year>2008</year>
          ).
          <article-title>Participatory budgeting in Europe: potentials and challenges</article-title>
          .
          <source>International Journal of Urban and Regional Research</source>
          ,
          <volume>32</volume>
          (
          <issue>1</issue>
          ),
          <fpage>164</fpage>
          -
          <lpage>178</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref19">
        <mixed-citation>
          <string-name>
            <surname>Skogerbø</surname>
            ,
            <given-names>E.</given-names>
          </string-name>
          , &amp;
          <string-name>
            <surname>Enli</surname>
            ,
            <given-names>G.</given-names>
          </string-name>
          (
          <year>2008</year>
          ).
          <article-title>Digitale dilemmaer: nye medieformer, nye utfordringer</article-title>
          . Oslo: Gyldendal akademisk.
        </mixed-citation>
      </ref>
      <ref id="ref20">
        <mixed-citation>
          <string-name>
            <surname>SSB</surname>
          </string-name>
          (
          <year>2017</year>
          ) https://www.ssb.no
          <article-title>/kultur-og-fritid/artikler-og-publikasjoner/fa-aktive-medlemmer-ipolitiske-partier</article-title>
        </mixed-citation>
      </ref>
      <ref id="ref21">
        <mixed-citation>
          <string-name>
            <surname>Steen-Johnsen</surname>
            ,
            <given-names>K.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Enjolras</surname>
            ,
            <given-names>B.</given-names>
          </string-name>
          , &amp;
          <string-name>
            <surname>Wollebaek</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          (
          <year>2013</year>
          ).
          <article-title>Sosiale medier, samfunnspolitisk deltagelse og kontroll</article-title>
          .
          <source>Internasjonal Politikk</source>
          ,
          <volume>71</volume>
          (
          <issue>2</issue>
          )
          <string-name>
            <surname>Saebø</surname>
          </string-name>
          , Ø.,
          <string-name>
            <surname>Rose</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          , &amp;
          <string-name>
            <surname>Nyvang</surname>
            ,
            <given-names>T.</given-names>
          </string-name>
          (
          <year>2009</year>
          ).
          <article-title>The Role of Social Networking Services in eParticipation Electronic Participation</article-title>
          (Vol.
          <volume>5694</volume>
          , pp.
          <fpage>46</fpage>
          -
          <lpage>55</lpage>
          ): Springer Berlin / Heidelberg.
        </mixed-citation>
      </ref>
      <ref id="ref22">
        <mixed-citation>
          <string-name>
            <surname>Templier</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          , &amp;
          <string-name>
            <surname>Pare</surname>
            ,
            <given-names>G.</given-names>
          </string-name>
          (
          <year>2015</year>
          ).
          <article-title>A Framework for Guiding and Evaluating Literature Reviews</article-title>
          (Vol.
          <volume>37</volume>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref23">
        <mixed-citation>
          <string-name>
            <surname>Zepic</surname>
            ,
            <given-names>R.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Dapp</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          , &amp;
          <string-name>
            <surname>Krcmar</surname>
            ,
            <given-names>H.</given-names>
          </string-name>
          (
          <year>2017</year>
          ).
          <article-title>Reasons for Low Participation in German Participatory Budgeting: A Public Administration Perspective 17th European Conference on Digital Government (ECDG</article-title>
          <year>2017</year>
          )
          <article-title>(pp</article-title>
          .
          <fpage>262</fpage>
          -
          <lpage>269</lpage>
          ). Lisbon, Portugal.
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>