<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>Designing and implementing a global multilingual real-time location-based game as a novel highly gamified spatial crowdsourcing platform</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Manuel F. Baer</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>Centre for eResearch, University of Auckland</institution>
          ,
          <country country="NZ">New Zealand</country>
        </aff>
      </contrib-group>
      <fpage>88</fpage>
      <lpage>109</lpage>
      <abstract>
        <p>Spatial crowdsourcing and other participatory methods of high-quality spatial data generation are increasingly popular in academic research. However, participatory approaches face issues of user motivation and retention. Gamification has been identified as a valuable approach towards increasing user motivation, engagement and retention by ofering an entertaining experience. This paper presents a case study of the literature-based development and implementation of Arcane Shift, a novel globally available locationbased game as a highly gamified spatial crowdsourcing platform. In particular, this paper focuses on the first three activities (problem identification, problem definition and application development) of the Design Science Research Methodology and how the implementation was guided by the established Gamification Framework for Volunteered Geographic Information and the Mechanics, Dynamics and Aesthetics Framework. Internal testing revealed the game to be enjoyable, albeit in need of further content. Four concrete implementation recommendations and considerations (develop for testing, incorporate caching, build for longevity and spatial is special) were gleaned from the development process and are presented as a methodological contribution to the design and development of location-based games and spatial crowdsourcing platforms. Finally, a list of future research agendas is provided including urban data generation, educational content delivery and motivating physical activity.</p>
      </abstract>
      <kwd-group>
        <kwd>eol&gt;Design science research</kwd>
        <kwd>mechanics</kwd>
        <kwd>dynamics</kwd>
        <kwd>aesthetics framework</kwd>
        <kwd>gamification framework for volunteered geographic information</kwd>
        <kwd>spatial crowdsourcing</kwd>
        <kwd>gamification</kwd>
        <kwd>location-based game</kwd>
      </kwd-group>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>1. Introduction</title>
      <p>
        Participatory approaches to generating
spatial data for academic research have enjoyed a
surge in popularity over the past two decades
[
        <xref ref-type="bibr" rid="ref1 ref2 ref3 ref4">1, 2, 3, 4</xref>
        ]. In particular, spatial crowdsourcing
approaches have shown immense potential in
generating rich high-quality spatial information
about our environments for comparatively low
ifnancial investments [
        <xref ref-type="bibr" rid="ref2 ref5 ref6 ref7">5, 6, 7, 2</xref>
        ]. However, a
limiting factor of crowdsourcing approaches,
and participatory approaches in general, is
user motivation and retention [
        <xref ref-type="bibr" rid="ref10 ref11 ref8 ref9">8, 9, 10, 11</xref>
        ].
A maturing approach to increasing user
engagement, retention and satisfaction in
crowdsourcing tasks is gamifying the participatory
process. Gamification, summarised as “hedonic
or entertainment-oriented technologies being
reappropriated for productive use” [12, p. 191],
has been applied to a number of crowdsourcing
8th International GamiFIN Conference 2024 (GamiFIN
2024), April 2-5, 2024, Ruka, Finland.
$ manuel.baer@auckland.ac.nz (M. F. Baer)
0000-0002-9474-3299 (M. F. Baer)
© 2024 Copyright for this paper by its authors.
      </p>
      <p>Use permitted under Creative Commons
License Attribution 4.0 International (CC BY</p>
      <p>
        CPWrEooUrckResehdoinpgs IhStpN:/c1e6u1r3-w-0s.o7r3g (4CC.0EE).UURR-WWS.oorrgk)shop Proceedings
initiatives (e.g. [
        <xref ref-type="bibr" rid="ref13 ref14 ref15 ref16">13, 14, 15, 16</xref>
        ]) and has shown
to successfully motivate users to participate
[
        <xref ref-type="bibr" rid="ref17 ref9">9, 17</xref>
        ]. However, many gamification eforts in
academic crowdsourcing have implemented a
limited number of gamified elements such as
points, badges and leaderboards [
        <xref ref-type="bibr" rid="ref17 ref18">18, 17</xref>
        ] whilst
highly gamified applications such as
locationbased games remain rare.
      </p>
      <p>
        Meanwhile, commercial location-based games
such as Ingress1 and Pokémon GO2 have
motivated millions of users [
        <xref ref-type="bibr" rid="ref19 ref20 ref21">19, 20, 21</xref>
        ] to contribute
data on unprecedented scales [
        <xref ref-type="bibr" rid="ref22 ref23">22, 23</xref>
        ].
Locationbased games ofer playful virtual interactions
according to the real-world coordinates of a
respective user [
        <xref ref-type="bibr" rid="ref21 ref24">24, 21</xref>
        ] and are predominantly
played in urban areas [
        <xref ref-type="bibr" rid="ref24 ref25">25, 24</xref>
        ] (showing
spatial biases towards wealthier neighbourhoods
[
        <xref ref-type="bibr" rid="ref19 ref26">26, 19</xref>
        ]).
      </p>
      <p>Skillfully balancing entertainment and
spatial crowdsourcing tasks has seen success in a
handful of academic studies [cf. 27, 13, 16], and
seeing the great potential of using more ludic
applications such as location-based games to
motivate users to contribute data towards
scien1https://www.ingress.com/ (accessed: 19.11.2023)
2https://pokemongolive.com/ (accessed:
19.11.2023)
tific crowdsourcing eforts, this paper presents
the design, development and implementation
of a globally available, modular, multilingual
real-time location-based game as an underlying
crowdsourcing platform. Specifically, this
paper aims to make a theoretical contribution to
the design and implementation of entertaining
spatial crowdsourcing platforms by presenting
the case study of Arcane Shift, a location-based
game as a spatial crowdsourcing platform. This
paper is particularly interested in how we can
design and implement a highly gamified spatial
crowdsourcing platform in the form of a
globally accessible location-based game by focusing
on the following research questions:
• What are the key underlying game
feature requirements to develop a highly
gamified spatial crowdsourcing platform
and how do these map onto established
implementation frameworks?
• How can underlying crowdsourcing
functionalities be complemented with
entertaining game features ofering global
realtime gameplay?
• What are key considerations and
recommendations for developing spatial
crowdsourcing platforms and location-based
games?</p>
      <p>In the following, I introduce relevant previous
work as well as the topical background before
diving into the specifics of the underlying
systems and how individual features were designed
and implemented. The final sections of the
paper culminate in discussing the implemented
location-based game in light of the literature,
presenting key lessons learnt during the design,
development and implementation processes as
a theoretical contribution to the design of
gamified spatial crowdsourcing platforms. Finally,
this paper presents further recommendations
and propositions as well as future avenues of
research.</p>
    </sec>
    <sec id="sec-2">
      <title>2. Previous work</title>
      <p>
        Multiple studies showcase the direct influence
our surroundings have on our social, mental
and physical well-being [
        <xref ref-type="bibr" rid="ref28 ref29 ref30">28, 29, 30</xref>
        ]. Examples
range from street design increasing perceived
safety [
        <xref ref-type="bibr" rid="ref31 ref32">31, 32</xref>
        ], over particular areas ofering
tranquillity [
        <xref ref-type="bibr" rid="ref33 ref34">33, 34</xref>
        ] boosting mental well-being
[
        <xref ref-type="bibr" rid="ref35 ref36">35, 36</xref>
        ], to public transportation networks
increasing physical activity [
        <xref ref-type="bibr" rid="ref37">37</xref>
        ] whilst reducing
personal motorised vehicles. However, much
of the data used to understand the status quo
of our environment are produced by sensors,
through computational algorithms or reported
by specific demographic groups. Temperature
and pollution readings are captured by fixed
measuring stations [
        <xref ref-type="bibr" rid="ref38">38</xref>
        ] or through remote
sensing [
        <xref ref-type="bibr" rid="ref39">39</xref>
        ], walkability analyses of urban and
residential areas focus on tangible variables of the
built environment disregarding individual
perceptions [
        <xref ref-type="bibr" rid="ref40">40</xref>
        ], and city planning is only now
gravitating from top-down approaches towards
the inclusion of local citizens [
        <xref ref-type="bibr" rid="ref41 ref42">41, 42</xref>
        ].
      </p>
      <sec id="sec-2-1">
        <title>2.1. Spatial crowdsourcing and participatory data collection</title>
        <p>
          Contrasting more automated spatial data
collection approaches are initiatives that include
non-expert users in spatial crowdsourcing tasks.
Many approaches exist concerning the
collection of spatial information contributed by
nonexperts including volunteered geographic
information [
          <xref ref-type="bibr" rid="ref5">5</xref>
          ] where individuals voluntarily make
some form of spatial data available [
          <xref ref-type="bibr" rid="ref1">1</xref>
          ] and
public participation geographic information systems
(PPGIS) [
          <xref ref-type="bibr" rid="ref43">43</xref>
          ] where individuals contribute data
through a participatory mapping efort. An
umbrella term for many participatory (spatial)
data collection approaches is crowdsourcing
[
          <xref ref-type="bibr" rid="ref1 ref3 ref4">1, 3, 4</xref>
          ], “a type of participative online
activity in which an individual, an institution, a
non-profit organization, or company proposes
to a group of individuals of varying knowledge,
heterogeneity, and number, via a flexible open
call, the voluntary undertaking of a task“ [44, p.
197]. This participatory approach can broadly
be split into two categories: (1) active
crowdsourcing where data is actively contributed for
a specific task or question as part of a data
collection efort (e.g. open street map (OSM)3),
and (2) passive crowdsourcing where data is
contributed for another intention than what it
is ultimately used for (e.g. social media data)
[
          <xref ref-type="bibr" rid="ref1">1</xref>
          ].
        </p>
        <p>
          Spatial crowdsourcing initiatives have been
found to produce high-quality data at a
fraction of the cost of more traditional
expertbased approaches [
          <xref ref-type="bibr" rid="ref1">1</xref>
          ] and have been found to
complement more technocratic and top-down
approaches to data collection (e.g. through
sensory networks and remote sensing) by
providing novel insights into more subtle
dimensions of human perception [cf. 45, 46, 33, 34].
        </p>
        <sec id="sec-2-1-1">
          <title>3https://www.openstreetmap.org/</title>
          <p>
            19.11.2023)
(accessed:
This can lead to a more holistic
understanding of emergent spatial issues such as
urbanisation, sustainability and well-being, making
spatial crowdsourcing an invaluable data
generation approach. Whilst successful large-scale
spatial crowdsourcing eforts in academia
remain scarce, non-academic entities have
identiifed the value of location-based user-generated
content and boast several successful
applications. Examples include the location-based
games Ingress and Pokémon GO crowdsourcing
the collection of culturally significant objects
through their Wayfarer system [
            <xref ref-type="bibr" rid="ref23">23</xref>
            ], Google
Maps collecting vast amounts of user-generated
spatial information such as points of interest
[
            <xref ref-type="bibr" rid="ref47">47</xref>
            ] and Waze focussing on mobility-related
data such as real-time trafic updates [
            <xref ref-type="bibr" rid="ref17">17</xref>
            ].
Despite having large repositories of spatial data,
these are not easily accessible, nor are these
systems customisable to specific scientific data
needs, and thus new academic initiatives are
called for.
          </p>
          <p>
            A shift towards more bottom-up approaches
to data collection in academic spatial research
encourages a reformulation of traditional
research agendas and has raised some underlying
methodological questions about how we can
effectively include citizens in science and how we
can motivate long-term engagement. However,
questions of user motivation and user retention
arise [
            <xref ref-type="bibr" rid="ref10 ref11">10, 11</xref>
            ], especially when competing with
an onslaught of entertainment and otherwise
available playful activities.
          </p>
        </sec>
      </sec>
      <sec id="sec-2-2">
        <title>2.2. Gamified crowdsourcing and location-based games</title>
        <p>
          A recent development sees the emergence of
crowdsourcing eforts capitalising on
entertaining features through gamification [
          <xref ref-type="bibr" rid="ref9">9</xref>
          ].
Gamiifcation can be summarised as “a process of
enhancing services with (motivational)
afordances in order to invoke gameful experiences
and further behavioral outcomes” [18, p. 3026].
Gamification shows great potential in
generating crowdsourced urban data and
complementing geospatial crowdsourcing more generally
[
          <xref ref-type="bibr" rid="ref2 ref9">9, 2</xref>
          ]. In the context of crowdsourcing,
gamiifcation has successfully been incorporated in
a number of academic studies as a means of
increasing user motivation (for an overview,
see [
          <xref ref-type="bibr" rid="ref9">9</xref>
          ]), albeit commonly implementing
simple competitive features such as points and
leaderboards. More gamified geospatial
crowdsourcing eforts include Urbanopoly [
          <xref ref-type="bibr" rid="ref27">27</xref>
          ], a
gamified urban data crowdsourcing efort and
FotoQuest Austria [
          <xref ref-type="bibr" rid="ref13">13</xref>
          ] as well as StarBorn [
          <xref ref-type="bibr" rid="ref16">16</xref>
          ],
entertaining approaches to collecting landcover
information.
        </p>
        <p>
          Location-based games are entertaining
applications, commonly played on devices with
location tracking features such as a global
positioning system, where a player’s in-game
position reflects their real-world coordinates [
          <xref ref-type="bibr" rid="ref24">24</xref>
          ].
Players navigate the virtual game world by
changing their real-world position and in-game
interactions are dependent on the real-world
position of a user and their proximity to virtual
objects. As such, location-based games ofer
a particularly interesting platform to collect
geospatial data from a heterogeneous group of
users. Pokémon GO and Ingress are highly
popular commercial location-based games [
          <xref ref-type="bibr" rid="ref21">21</xref>
          ]
shadowing other approaches in terms of user
engagement, successfully motivating hundreds
of millions of users to engage with the
application and contribute rich spatial information
[
          <xref ref-type="bibr" rid="ref22 ref23">22, 23</xref>
          ]. This begs the question: can a
locationbased game be implemented to crowdsource
heterogeneous spatial information for research
purposes?
        </p>
      </sec>
      <sec id="sec-2-3">
        <title>2.3. Design science research and relevant frameworks</title>
        <p>
          Application development and implementation
processes concerning the construction of
information systems as problem solutions to
concrete issues can be grounded in design science
research [
          <xref ref-type="bibr" rid="ref48 ref49 ref50 ref51">48, 49, 50, 51</xref>
          ] which "seeks to enhance
technology and science knowledge bases via the
creation of innovative artifacts that solve
problems and improve the environment in which
they are instantiated" [51, p. 1], or more
generally "the construction and evaluation of generic
means–ends relations" [50, p. 470]. One widely
used and well-established framework is the
Design Science Research Methodology presented
by [
          <xref ref-type="bibr" rid="ref49">49</xref>
          ], dividing the overall goal of
developing and evaluating an application aiming to
address a specific problem statement into six
distinct activities (for a detailed discussion, see
[
          <xref ref-type="bibr" rid="ref49">49</xref>
          ]):
1. Identifying concrete research problems
as well as justifying the importance of
the proposed solutions
2. Defining specific objectives and
requirements of the envisioned solutions
3. Designing and implementing the
application or artefact
4. Demonstrating the implemented
application’s or artefact’s ability to address the
identified problems
5. Evaluating the application’s or artefact’s
ability to address the identified problems
6. Communicating the initial problem
statements, the implemented application or
artefact and the design process as well
as the evaluation to a heterogeneous
audience
        </p>
        <p>
          Whilst ofering a stable foundation to
conduct research relating to application design
and ofering clearly defined overarching
activities, it makes sense to complement the Design
Science Research Methodology with additional
more specific frameworks when developing a
specific application. To build a globally
available, modular, multilingual crowdsourcing
platform as a location-based game, further
established frameworks exist to ground specific
development decisions in theory and guide the
implementation. Two established frameworks
with increasing topic specificity include the
Mechanics, Dynamics, Aesthetics Framework [
          <xref ref-type="bibr" rid="ref52">52</xref>
          ]
concerning general gamified applications and
the Gamification Framework for Volunteered
Geographic Information [
          <xref ref-type="bibr" rid="ref53">53</xref>
          ] which focuses on
volunteered geographic information, albeit
applicable to geospatial crowdsourcing eforts in
general.
        </p>
        <p>
          A framework seeing widespread adoption in
both designing [
          <xref ref-type="bibr" rid="ref15">15</xref>
          ] as well as analysing [
          <xref ref-type="bibr" rid="ref54">54</xref>
          ]
games and gamified applications is the
Mechanics, Dynamics and Aesthetics framework
[
          <xref ref-type="bibr" rid="ref52">52</xref>
          ] which segregates an application into three
underlying categories. Firstly, the aesthetics
of an application is what the user directly
experiences and what invokes a user’s emotional
response towards specific parts of the
application (e.g. the design of the user interface).
An application’s aesthetics are of particular
importance and can further be subdivided into
eight categories: sensation, fantasy, narrative,
challenge, fellowship, discovery, expression, and
submission. Secondly, the dynamics represent
the systems that influence or are influenced
by a player’s interaction with the application
and which result in specific behavioural
outcomes (e.g. a button that opens a specific
screen afording new interactions within the
application). Lastly, the underlying algorithms
and data structures are summarised within the
mechanics of an application and build the
backbone of the gamified system (e.g. the database
in which application data is stored). What
makes the Mechanics, Dynamics and
Aesthetics framework particularly interesting in terms
of gamified application design is the emphasis
on taking both the developers as well as the
user’s perspectives into consideration,
allowing for a more holistic approach to application
design.
        </p>
        <p>
          Drilling down to specific frameworks
regarding the gamification of geospatial information
collection, the Gamification Framework for
Volunteered Geographic Information aims at
bridging the divide between geospatial
crowdsourcing applications and their gamified equivalent
by mapping the terminology between the two
domains [
          <xref ref-type="bibr" rid="ref53">53</xref>
          ]. When translating from a
native volunteered geographic information
application to a gamified implementation, users
become players, the map transforms into the
game board, geo-data is used to generate virtual
goods and tasks translate to challenges.
Additional gamification elements aim at increasing
collaboration and/or competition and include
leaderboards, social elements such as friends
lists, votes as well as levels, points, badges and
bonuses as typical rewards.
        </p>
        <p>This paper delves particularly into the first
three activities of the design science research
approach. After having identified the concrete
research problem of increasing user motivation
and retention in spatial crowdsourcing in the
introduction and background sections (activity
1), in the following I focus on defining specific
objectives and requirements of a solution
(activity 2) and shed particular light on how the
Mechanics, Dynamics and Aesthetics
framework as well as the Gamification Framework
for Volunteered Geographic Information were
used to develop and implement the application
Arcane Shift (activity 3).</p>
      </sec>
    </sec>
    <sec id="sec-3">
      <title>3. Design and implementation</title>
      <p>
        The location-based game named Arcane Shift
aims to combine various game elements into
an entertaining crowdsourcing experience. In
a first step, and using the two aforementioned
frameworks, a list of envisioned features was
created to guide the development. Firstly, the
overarching theme of the application was set
as a synthwave-inspired sci-fi experience where
users would captain a futuristic spacecraft.
Setting the theme allowed for individual game
features to be designed around said theme.
To allow for persistent player data as well as
to collect basic demographic information from
the participants, crucial data in active
crowdsourcing platforms, a registration system and
onboarding system were needed. Seeing the
location-based nature of the application, an
underlying spatial system was needed to allow the
generation and visualisation of location-based
content, a common feature of location-based
games [
        <xref ref-type="bibr" rid="ref21 ref23 ref24">24, 21, 23</xref>
        ]. Tying into the spatial
system of the location-based game, the application
required a number of spatial content systems
(e.g. resource system, building system) to
allow for the persistence and correct placement
of in-game content such as items and
structures. To introduce a form of progression and
to reward the user for in-game activities, a
level system was required which is commonly
found in gamified applications [
        <xref ref-type="bibr" rid="ref12 ref53">53, 12</xref>
        ]. Seeing
the application focuses on spatial
crowdsourcing, a questing system was needed allowing
the integration of a variety of crowdsourcing
tasks. Primarily, the application should allow
users to upload location-specific content such
as location-based natural language reports and
ratings, commonly collected variables in spatial
crowdsourcing. To foster motivation and
incentivise continued engagement with the platform,
the application was in need of entertaining
systems ofering a variety of in-game activities. In
line with the overarching theme of the
application, a ship upgrade system, an attack system
and a forge system were envisioned. Having
unlockable and upgradable in-game items ofers
a form of progression and positively afects a
user’s motivation [
        <xref ref-type="bibr" rid="ref12 ref53">53, 12</xref>
        ]. Attackable in-game
structures increase an application’s competitive
elements, which have been found to positively
afect user engagement [
        <xref ref-type="bibr" rid="ref17">17</xref>
        ] and allowing users
to forge new items ofers a form of discovery
and a collaborative element, increasing the
likelihood that the application is recommended
[
        <xref ref-type="bibr" rid="ref17">17</xref>
        ]. Lastly, an alert system was deemed
necessary to inform users of important in-game
activities.
      </p>
      <p>These envisioned features of the application
were mapped to the broader categories of the
two aforementioned frameworks helping to
structure the development and guide the
implementation. Table 1 shows the implemented features
of the location-based game and the
corresponding categories of the frameworks.</p>
      <sec id="sec-3-1">
        <title>3.1. The core game loop</title>
        <p>Within the location-based game, users captain
a futuristic spacecraft (Figure 1 H ) and initially
collected basic ores that were procedurally
generated depending on the underlying real-world
land cover of an area (Figure 1 G). Users could
either combine specific combinations of ores
into new materials increasing their value or sell
ores for gold rewards. Gold could then be used
to construct buildings for their team (Figure
1 D) or to buy other ores. If a user’s ship had
enough energy (Figure 1 I ), enemy buildings
(Figure 1 E ) could be attacked (Figure 1 F )
and destroyed freeing up an area for friendly
buildings. Further, users could accept and
complete a variety of quests (individual
crowdsourcing tasks) from survey drones (Figure 1 B) or
their quest log (Figure 1 J ). Many in-game
activities such as completing quests, collecting
resources, forging items and building friendly
or destroying enemy buildings rewarded
experience points (Figure 1 I ). Leaderboards showed
the top-performing users and an activity log
(Figure 1 L) displayed a user’s past actions
as well as the enemy users who attacked their
buildings. By limiting a player’s view to a fixed
distance (approximately a radius of 1km) the
game aimed to engage the users’ inquisitive
nature and hoped to motivate users to explore
their surroundings.</p>
        <p>First user feedback revealed the core game
loop to be interesting, albeit lacking content to
motivate long-term engagement. As such
leveldependent unlockable content was introduced.
Thereafter, users could additionally spend gold
on unlocking various spacecraft component
upgrades such as energy generators (e.g.
increasing energy regeneration rate), tractor beams
(e.g. increasing number of simultaneous ore
collections) and weapons (e.g. increasing
damage). Gold could also be spent on unlocking
and constructing improved buildings for the
user’s team such as forges with reduced forge
times or markets with increasingly discounted
prices. An overview of the current core game
loop is presented in Figure 2.</p>
        <p>In the following, I first present the
infrastructure with which the application was built before
introducing core features as well as discussing
the implemented systems.</p>
      </sec>
      <sec id="sec-3-2">
        <title>3.2. Infrastructure and core systems</title>
        <p>The location-based game was built from scratch
in the Unity4 game engine (version: 2022.2).</p>
        <p>4https://unity.com/ (accessed: 10.01.2024)
sys- virtual goods,</p>
        <p>lenges
Game Feature
registration
and
onboarding
spatial system
level system
ship system
quest system
forge system
building system
attack system
activity alerts</p>
        <p>Mechanics, Dynamics, Aesthetics Framework
Gamification
Framework for Volunteered
Geographic
Information
user / player aesthetics: narrative, fantasy; dynamics:
registration form, email confirmation, demographic
information report, character selection, team selection;
mechanics unity game code, firebase
authentication, firestore database
map / gameboard aesthetics: discovery; dynamics: providing and
translating coordinates, downloading, caching and
displaying map tiles; mechanics: unity game code,</p>
        <p>MapBox tile server
challenges, levels aesthetics: challenge dynamics: increasing levels
unlocks various in-game items mechanics: unity
game code, firestore database
virtual goods, chal- aesthetics: fantasy, narrative, challenge, discovery;
lenges, levels dynamics: ship components can be unlocked and
equipped changing the behaviour of a user’s ship;
mechanics: unity game code, firestore database
tasks / challenges, geo- aesthetics: challenge, discovery, expression;
dydata namics: quests can be accepted and submitted,
submitting quests rewards the user; mechanics:
unity game code, firestore database
chal- aesthetics: fantasy, narrative, challenge, discovery;
dynamics: spawning resources depend on the
realworld landcover in the spawn location, resources
can be collected and forged into new resources,
collecting resources rewards experience points,
resources can be bought and sold; mechanics: unity
game code, firestore database
virtual goods, chal- aesthetics: sensation, challenge, discovery;
dylenges namics: forging allows new items to be
discovered increasing their value, forging rewards
experience points; mechanics: unity game code, firestore
database
gameboard, social ele- aesthetics: fantasy, challenge, fellowship,
expresment sion; dynamics: buildings can be built and
interacted with by users on the same team. Enemy
buildings can be attacked and destroyed;
mechanics: unity game code, firestore database
gameboard, points, lev- aesthetics: sensation, fantasy, challenge,
fellowels, challenges ship, discovery; dynamics: in-game buildings of the
enemy team can be attacked and the attack
damage depends on a user’s ship’s weapon, destroying
a building rewards experience points; mechanics:
unity game code, firestore database
social element aesthetics: discovery, fellowship, challenge;
dynamics: alerts of important in-game activities;
mechanics: unity game code, firestore database
Google Firebase5 and Firestore6, massively
scal</p>
        <p>5https://firebase.google.com/
10.01.2024)</p>
        <p>6https://firebase.google.com/docs/firestore
(accessed: 10.01.2024)
able NoSQL database solutions, were
introduced to provide authentication and storage for
the application. Leveraging Firestore’s event
functionality I introduced real-time gameplay
through subscribing to database document events
and acting accordingly. In other words,
updating a document in the database triggers an
update event, which in turn triggers all
listening devices to call a respective method. This
event-based approach allows for chosen in-game
actions to be propagated to all connected user
devices, allowing for real-time gameplay.</p>
        <p>
          For example: When a user attacks an enemy
building the persisting building data in Firestore
is updated, triggering all listening devices to call
the method to update the respective building’s
health.
3.2.1. Data downloading and caching
In a first iteration, all game data was stored on
the user’s device, however, to allow for rapid
development and balancing during testing, game
content must remain updatable allowing for
small tweaks according to user feedback (e.g.
weapon damage). Seeing the uneven
coverage of mobile broadband internet, especially
in rural areas [
          <xref ref-type="bibr" rid="ref55">55</xref>
          ], and to limit needed
mobile data, specific game content is cached on
a user’s device. A caching pipeline was
implemented where content is initially stored on
a server and is only (re)downloaded from the
server if the content is not found in the user’s
persistent data cache or if the content has been
updated on the server. The downloaded
content is then stored in a persistent cache on the
user’s device as well as in memory, allowing
future access to up-to-date data without needing
to (re)download the content (cf. Figure 3).
3.2.2. Application environments
At the start of the development process, a
single environment was used for development and
testing. However, with growing complexity and
to allow for easy updates and continuous
integration [
          <xref ref-type="bibr" rid="ref56 ref57">56, 57</xref>
          ], the application was segregated
into three separate environments: a
development environment for rapid prototyping, a
staging environment for beta-testing with test users
and a production environment used for the live
application. Each environment is accompanied
by a decoupled database instance, allowing for
large changes to be implemented and tested
without afecting other environments. This
approach follows general software development
guidelines and allows for rapid prototyping,
continuous integration and stable production
environments.
3.2.3. Internationalisation
Having multilingual content allows for a more
diverse audience and has been found to be an
important consideration in modern applications
[
          <xref ref-type="bibr" rid="ref58 ref59">58, 59</xref>
          ]. Internationalisation of game content
was thus implemented as an integral part of the
application and allows the game to be played
in multiple languages (at the time of writing,
this includes te reo Ma¯ori, English, French and
German). Chosen game content (e.g. textual
content) was translated into the mentioned
languages and respective translations were
initially stored using Unity’s Localization package.
        </p>
        <p>However, similar to the reasoning of moving
game content data to an external database to
allow for rapid prototyping and quick
adjustments (Section 3.2), translation tables were
moved to the Firestore database. In the
current iteration, translated texts are referenced
as translation keys and translation data is
handled similarly to general game data in terms
of caching where data is only downloaded if
it is not found in the translation table cache
on a user’s device, or if any updates to the
translation table are detected. In other words,
translation tables for textual content are stored
online and downloaded on demand and only
if the database document has been updated.</p>
        <p>
          This allows the textual content to be seamlessly
translated and updated in the application, a
key ingredient in contemporary application
design [
          <xref ref-type="bibr" rid="ref60">60</xref>
          ].
        </p>
      </sec>
      <sec id="sec-3-3">
        <title>3.3. Registration and onboarding</title>
        <p>
          To ensure an individualised experience, users
must register and go through an onboarding
process (Figure 4). After registering with email
and password, users are required to confirm
their email address to minimise misuse. When
logging in for the first time, an onboarding
process is started, where users are first presented
with the free and prior informed consent of
the application’s intended use and how a user’s
data will be used and processed. This allows
users to make an informed decision on whether
they would like to proceed, in line with
contemporary data protection regulations when
collecting user-generated data [
          <xref ref-type="bibr" rid="ref61">61</xref>
          ]. Users are
then introduced to the underlying narrative
of the location-based game and are asked to
make a character, provide basic demographic
information and choose their preferred
username and team. The onboarding process ties
into the game narrative, which has been found
to increase user engagement and motivation
[
          <xref ref-type="bibr" rid="ref62">62</xref>
          ]. Once a user has provided the required
information, they enter the location-based game
world.
        </p>
      </sec>
      <sec id="sec-3-4">
        <title>3.4. Spatial features</title>
        <p>
          A defining feature of location-based games is
the translation of a user’s real-world
coordinates into a (virtual) game world [
          <xref ref-type="bibr" rid="ref24">24</xref>
          ]. This
generally requires two broader systems: (1)
querying a user’s real-world coordinates as well
as (2) translating these to respective
gameworld coordinates and constructing the game
world accordingly. At first, the location-based
game was developed implementing the MapBox
Software Development Kit (SDK)7 for Unity.
However, the SDK was accompanied by three
main limitations: (1) computational overhead
due to the number of features, (2) complexity
of extending the SDK and (3) discontinued
development of the SDK leading to deprecated
third-party code. As such, two custom spatial
systems were built to allow customisation and
integration in the game.
        </p>
        <sec id="sec-3-4-1">
          <title>7https://www.mapbox.com/unity</title>
          <p>05.09.2023)
(accessed:</p>
          <p>In the implemented application, a location
provider was implemented which uses a
respective device’s location features to periodically
query the user’s current coordinates. A
mapping engine was built to leverage the location
provider and convert between the retrieved
realworld latitude and longitude coordinates and
the game world coordinates. The mapping
engine calculates the current map extent of
the player and identifies the needed map tiles.
The device’s cache is queried and existing map
tiles within the user’s vicinity are retrieved
and drawn to the screen as an underlying base
map in the game world. Missing map tiles are
downloaded from MapBox8 and cached on the
user’s device to reduce the bandwidth needed
when a user revisits locations. When the user
changes their location in the real world their
position within the game is updated to reflect
the player’s new real-world position. This in
return triggers the querying, downloading and
displaying of new relevant tiles as well as
disabling no longer needed tiles. Another key
implemented spatial feature of the mapping
engine is the support for the global hexagonal
indexing system H39. This allows chosen
content (e.g. quests, buildings, resources) to be
placed in hexagonal tiles on a global grid.</p>
        </sec>
      </sec>
      <sec id="sec-3-5">
        <title>3.5. Level system</title>
        <p>
          An important dimension in gamified
applications is progression and inducing a sense of
8https://www.mapbox.com/
04.01.2024)
9https://h3geo.org/ (accessed: 03.01.2024)
long-term in-game goals [
          <xref ref-type="bibr" rid="ref12 ref53">53, 12</xref>
          ]. The
implemented platform features experience points and
a levelling system, belonging to the most
common gamification elements [
          <xref ref-type="bibr" rid="ref18">18</xref>
          ]. Users initially
start the game at level 1 and are rewarded with
experience points for several in-game activities
such as the completion of quests, destroying
enemy buildings, building friendly buildings
and collecting or forging ores (cf. Figure 2).
Once a level-specific threshold of experience
points is reached, the player’s level is
incremented. Initially, the application featured a
linear progression system, where every level
required the same amount of accumulated
experience points to advance. However, in a
later iteration, this was changed to
exponential experience point thresholds to ensure
longterm challenges. With increasing player levels
new game content is unlocked, providing an
additional incentive to accumulate experience
points.
        </p>
      </sec>
      <sec id="sec-3-6">
        <title>3.6. Ship system</title>
        <p>Gaining increasing levels within the
application unlocks further in-game content. In an
initial version of the application, additional game
content was restricted to additional buildable
structures. However, user feedback highlighted
the lack of incentives and thus a ship
component system was introduced, where users
could upgrade individual components of their
spacecraft. Implemented ship components
consisted of energy generators, tractor beams and
weapon systems. The energy generator
component provides a limited amount of energy
that recharges over time and which limits the
allowed use of other components. Upgrades to
the energy generator include a higher energy
capacity and a faster recharge over time. The
tractor beam component uses energy to collect
resources. Upgrades include faster resource
collection times and an increasing number of
simultaneous collections. Finally, the weapon
system uses energy to fire projectiles at enemy
structures. Key upgrades include increasing
the number of simultaneous projectiles and
increasing each projectile’s damage. Users can
view key statistics as well as access their ship’s
cargo bay, ship components, a general market
and the building tool through the ship menu
(Figure 5).</p>
      </sec>
      <sec id="sec-3-7">
        <title>3.7. Quest system</title>
        <p>
          According to the Gamification Framework for
Volunteered Geographic Information, spatial
data contribution tasks in crowdsourcing are
mapped to challenges in gamified systems [
          <xref ref-type="bibr" rid="ref53">53</xref>
          ].
These are commonly implemented as quests,
short in-game tasks that aford progression in
the game, and commonly reward a player with
virtual in-game items or points. The quests
implemented in Arcane Shift fall into the broad
categories of designed or generated. The former
refers to quests that are purposefully designed
and placed at specific locations to collect
specific location-dependent information. In
contrast, the latter refers to crowdsourcing tasks
that are randomly generated throughout the
game world, independent of location. In a
preliminary version of the application, each
user could complete each quest once. However,
many spatial crowdsourcing tasks are suited
to be repeated in diferent locations (e.g.
perceived scenicness of an area) or at varying times
(e.g. perceived safety during the day vs.
during the night). Seeing the global nature of,
and the focus on, geospatial crowdsourcing,
quests were thus extended to be either local or
global and repeatable or non-repeatable. Table
2 shows an overview of the quest categories and
examples of questions or tasks that could be
incorporated.
        </p>
        <p>Quests are comprised of one or multiple quest
steps, which in return are made up of one or
multiple quest tasks (Figure 6). In addition,
quest steps can be locked and only made
available to the user if other specific quest steps are
completed first. This allows a sequence of quest
steps to be combined into a journey, guiding
the user through multiple locations in the real
world. This in return opens the door to
collecting specific spatially nuanced information
or guiding users through educational journeys
in a given landscape.</p>
        <p>Example: A quest could ask a user to go to
the main train station and report on their
experience. This quest would be divided into two
quest steps (1) navigate to the specific area of
the train station and (2) report on their
experience, where (2) only becomes available after
completing (1).</p>
      </sec>
      <sec id="sec-3-8">
        <title>3.8. Resource system</title>
        <p>Collectable entities are common in
locationbased games and provide an incentive to
navigate the game world as well as a means of
generating in-game value. Within Arcane Shift
users can collect a variety of basic ore crystals
which are procedurally generated depending
on the respective real-world land cover of a
given area (Figure 1 G). Collected basic ore
crystals are stored in the users inventory and
can be sold for in-game gold, a basic in-game
currency. This currency can be used to buy
specific in-game ores as well as to unlock a
variety of in-game content (e.g. ship component
upgrades), supposing all level requirements are
Quests that are purposefully placed and that Complete the educational
can be completed exactly once in each avail- coast walk and do the
able location. quiz.</p>
        <p>Quests that are purposefully placed and that Is any of the
infrastruccan be repeated multiple times in each avail- ture in this park in need
able location, requiring a set amount of time of repair?
to pass between submissions.</p>
        <p>Quests that are purposefully placed and that Describe your favourite
can be completed exactly once globally. place in the world.
Quests that are purposefully placed and that Tell us about your
can be repeated multiple times globally, re- favourite new place you
quiring a set amount of time to pass between have discovered this
submissions. month.</p>
        <p>Quests that are randomly generated and What is your first
impresthat can be completed exactly once in each sion of this area?
available location.</p>
        <p>Quests that are randomly generated and How safe do you feel in
that can be repeated multiple times in each this area?
available location, requiring a set amount
of time to pass between submissions.</p>
        <p>Quests that are randomly generated and Do you prefer urban or
ruthat can be completed exactly once globally. ral landscapes?
These quests are useful to generate generic
not necessarily spatial data.</p>
        <p>Quests that are randomly generated and Upload an image of
that can be repeated multiple times glob- your favourite place this
ally, requiring a set amount of time to pass month.
between submissions.
met.</p>
      </sec>
      <sec id="sec-3-9">
        <title>3.9. Forge system</title>
        <p>
          Exploration as a dimension of gamification
ties into the Mechanics, Dynamics,
Aesthetics Framework aesthetic of discovery and has
been found to increase user motivation [
          <xref ref-type="bibr" rid="ref63 ref64">63, 64</xref>
          ].
Besides exploring the game world, another
implemented form of exploration revolves around
the forging feature, where players can submit
four ore crystals and have the chance to
discover new items (Figure 7). However, only
specific combinations of ores will result in a
successful forge rewarding a new item, meaning
players must explore the various combinations
and potentially share their findings with other
players.
        </p>
      </sec>
      <sec id="sec-3-10">
        <title>3.10. Building system</title>
        <p>
          Another key dimension is collaboration, which
has been found to boost motivation in and
engagement with an application as well as the
likelihood of word-of-mouth recommendations
[
          <xref ref-type="bibr" rid="ref17 ref65">65, 17</xref>
          ]. Within Arcane Shift, the users must
choose between one of three teams during the
onboarding process. The teams each boast
an intriguing backstory and players within a
team can interact with buildings built by other
players within the same team. Buildings of
the same team allow for more in-game
interactions (e.g. access to the forging mechanic
as shown in Figure 7) and as such, there is a
strong incentive for players to be in proximity
to friendly buildings. In a first iteration, users
could build buildings if they had suficient gold
in their inventory and if there was an
unoccupied hexagonal H3 tile in the vicinity of the
user. The application would display preview
buildings and once a user chose the preferred
building location by clicking on a preview, a
loading screen would show whilst the
building was created and subsequently saved on the
server. However, constant loading screens were
perceived as disruptive to the game experience.
As such, loading screens were masked with a
new building warp mechanic, where the
application would show a warping animation instead
of a loading screen (Figure 8). In addition,
further unlockable building tiers were added as
additional game content.
        </p>
      </sec>
      <sec id="sec-3-11">
        <title>3.11. Attack system</title>
        <p>
          Competition, if provided in adequate amounts,
has been found to increase user motivation and
engagement in games and gamified applications
[
          <xref ref-type="bibr" rid="ref17 ref9">9, 17</xref>
          ]. As such, providing the right amount of
competitive elements is a valuable tool within
gamification eforts. Within the location-based
game, besides incorporating a variety of
leaderboards, competition is fostered by the limited
building space in the virtual world and enemy
buildings being destroyable to make room for
buildings for one’s own team. If a user’s ship’s
generator has enough energy, a user can click on
an enemy structure to fire the ship’s weapons at
the enemy structure, inflicting weapon-specific
damage to the structure. Once a structure’s
health depletes, it is destroyed to make room for
buildings for one’s own team, leading into the
dimension of competition. Building updates
must be propagated to all other players with
view of a specific building. This was achieved
by: (1) starting a weapon charging animation
to mask client-server communication lag, (2)
updating the persistent building data in the
database, (3) firing a document update event
informing all connected devices of a change and
(4) updating a respective building’s health on
all connected devices.
        </p>
      </sec>
    </sec>
    <sec id="sec-4">
      <title>4. Internal user testing</title>
      <p>A testing pipeline was set up on the Google
Playstore to perform internal user testing. This
allowed invited test participants to download
the application through the Google Playstore
and update the application as soon as new
versions of the application became available, a
crucial feature to allow for iterative
development. First internal pilot tests were conducted
with a small group of three users to test the
stability of the main game systems and to identify
points of improvement. All users were between
the ages of 32 - 45 years old and covered all
levels of expertise with location-based games
including one user with no experience, one
casual user and one expert user. The users were
given access to the application and received
an overview of the location-based game, the
various features of the application as well as
detailed instructions of the testing process. The
internal test users were asked to pay particular
attention to the core systems (e.g. questing,
collecting, building, attacking) and were asked
to report if they encountered any errors or
unexpected behaviour. In addition, the users were
asked for their opinion of the first test version
and potential improvements that would make
the game more enjoyable. Feedback was
collected through an online form, as voluntarily
submitted additional written feedback and as
one-on-one discussions.</p>
      <p>The users generally enjoyed the
locationbased game and reported it being engaging
and mostly stable (n = 3). The users reported
liking (n = 1) or loving (n = 2) the overall
aesthetics of the game and it being a little (n = 1),
very (n = 1) or extremely (n = 1) motivating in
its current state. Chosen features of the game
such as upgrading ship components, gaining
experience points and levelling up were reported
to be enjoyed by the test users (n = 3).
However, many features such as collecting resources,
constructing buildings and buying items from
the in-game market were reported to be only
moderately enjoyed due to the limited amount
of in-game content. Further key feedback
revolved around user convenience and interaction
features such as auto-login or revising the touch
detection and tap interactions within the
application. The location-based game will be
trialled with a larger test user group before
being opened up for public beta testing.</p>
    </sec>
    <sec id="sec-5">
      <title>5. Discussion and outlook</title>
      <p>The developed and implemented location-based
game is a novel highly gamified crowdsourcing
and citizen science platform, upon which
various interesting systems can be built. This
paper presents the implementation and the
literature-based reasoning behind various
design decisions as well as gives an overview of
the current state of the application and
future potential. In the following, I first discuss
gamifying spatial crowdsourcing in general
before presenting key recommendations for
building highly gamified spatial crowdsourcing
platforms and location-based games, gleaned from
lessons learnt during the development of
Arcane Shift. Finally, I lay out specific future
agendas for the platform and summarise the
ifndings in the conclusions.</p>
      <sec id="sec-5-1">
        <title>5.1. Gamifying crowdsourcing through a location-based game</title>
        <p>
          Common approaches implementing the design
science research method identify a specific
topical research question or organisational problem,
which is subsequently distilled into application
features to be implemented [
          <xref ref-type="bibr" rid="ref48 ref49 ref50 ref51">48, 49, 50, 51</xref>
          ]. The
approach presented here attempts to address
the more general problem of how users can be
motivated to participate in spatial
crowdsourcing tasks through a gamified application, in
return allowing a plethora of domain-specific
research questions to be asked. Specifically,
Arcane Shift was built to allow for a variety
of user contributions, allowing for most
crowdsourcing tasks to be translated into
locationbased game quests, ofering a novel approach
to crowdsourcing and citizen science.
        </p>
        <p>
          The implemented location-based game
further transcends traditional crowdsourcing
platforms in regards to the special focus on
entertaining and motivational elements,
allowing for high-quality data generation whilst
offering entertaining virtual experiences. This
contrasts with many more traditional
crowdsourcing approaches, where participants are
motivated through monetary rewards,
scientific or other forms of acknowledgements or are
encouraged to participate as part of a
curriculum [
          <xref ref-type="bibr" rid="ref1 ref10 ref2 ref9">1, 9, 10, 2</xref>
          ]. Scientific inquiries leveraging
crowdsourced spatial data are plentiful with
examples ranging from characterising specific
landscapes [
          <xref ref-type="bibr" rid="ref66">66</xref>
          ], over detecting popular
viewports [
          <xref ref-type="bibr" rid="ref67">67</xref>
          ] and analysing urban mobility [
          <xref ref-type="bibr" rid="ref68">68</xref>
          ], to
mapping a city’s smellscape [
          <xref ref-type="bibr" rid="ref45">45</xref>
          ]. These
generally use the rich spatial information contained
within geolocated social media data as well as
other forms of (online) user-generated content
and are a welcomed source of easily
accessible data from a heterogeneous crowd.
However, such datasets are generally noisy with
many irrelevant data points. The implemented
location-based game aims at minimising
irrelevant contributions through an active approach
to collecting data. Being able to strategically
place crowdsourcing tasks (quests) at specific
locations and the ability to create tailored tasks
to collect any information needed makes the
implemented application a novel highly
gamiifed crowdsourcing platform, transcending more
passive approaches to generating crowdsourced
data.
        </p>
        <p>
          Combining the ease of adding new
crowdsourcing tasks either globally or in specific
locations with the general goal of developing
extendable core systems makes this platform
especially interesting for longitudinal studies.
As such, this surpasses general academic
crowdsourcing eforts which are typically developed
for a specific project after which they tend
to disappear [
          <xref ref-type="bibr" rid="ref69">69</xref>
          ], potentially due to the
dififculty of updating the underlying systems to
accommodate new research endeavours. The
location-based game presented in this paper
takes a diferent approach: the core systems
are built to function independently of
overarching projects and research questions.
        </p>
      </sec>
      <sec id="sec-5-2">
        <title>5.2. Recommendations and lessons learnt</title>
        <p>
          During the implementation of Arcane Shift the
application has gone through several versions
and substantial amounts of code were
refactored with each new iteration. This dynamic
and evolving nature of the application is in
line with modern agile software engineering
approaches [
          <xref ref-type="bibr" rid="ref70 ref71">70, 71</xref>
          ] and allows for flexibility in
design and development. In the following, I
touch upon four key areas of location-based
game design that have proven to be key
improvements or important considerations in the
design and implementation process.
5.2.1. Develop for testing, test for
        </p>
        <p>development
Rapid prototyping and user testing are the
backbones of creating new information systems
and have proven to be invaluable in the
development of Arcane Shift. Developing applications
can be seen as an iterative circular process
starting with implementing features, subsequently
testing features and collecting feedback and
finally refactoring or implementing new features
according to the collected feedback, efectively
starting a new iteration loop. This approach
however requires two key implementation
considerations from the start of a project.</p>
        <p>
          Firstly, a given application should be divided
into at least three distinct environments: (1)
a development environment in which
development takes place, (2) a staging environment
with which internal or open user testing is
conducted and (3) a production environment
encompassing the live application. These
environments can further be extended according
to the needs of the project (e.g. additional
internal, closed and open beta testing
environments). Using modern versioning software
such as GIT10, the code base for each
environment can be tracked and changes can be
merged as required. Finally, each environment
should incorporate a decoupled database
instance so changes to one environment’s
backend database does not influence the other
environments. Seperating environments and
using version control is common practice in
software development [
          <xref ref-type="bibr" rid="ref71 ref72">71, 72</xref>
          ] and despite the
additional overhead of setting up versioning and
distinct environments with decoupled database
instances, this has proven invaluable to allow
for simultaneous testing of multiple versions as
well as continued development.
        </p>
        <p>
          A second important consideration is
making content easily updatable and propagating
updates to test users. This is especially
important for data where minor changes might
become necessary, for example for balancing
(e.g. weapon damage, energy use, recharge
rates). Storing such content online and
having the application download updated values
when the application starts allows developers
to change values as needed without needing to
distribute a new application build to the test
10www.git-scm.com (accessed: 03.01.2024)
users. However, downloading content on the
lfy has the caveat of needing an internet
connection, a particularly important consideration
when designing mobile applications.
5.2.2. Incorporate caching and on-demand
content retrieval
Moving content to online storage and
downloading needed data when starting an application
allows for rapid value updates, however,
loading content from external sources can result in
notable amounts of mobile data usage.
During the development of Arcane Shift it became
evident that the majority of content remained
unchanged between application usages. This
highlighted the importance of caching content
on the device to minimise mobile data usage.
Caching was introduced where content was only
downloaded from an external source if the
online content was updated. This allowed the
seamless updating of existing content as well as
the addition of new content, whilst minimising
needed bandwidth and mobile data usage. This
is in line with modern online application design
best practices, advocating for fast content
loading and easy navigation [
          <xref ref-type="bibr" rid="ref73 ref74">73, 74</xref>
          ]. Carefully
considering which content is immutable and which
content requires dynamic updating is thus a
key consideration when building location-based
games or spatial crowdsourcing applications.
5.2.3. Build for longevity
The location-based game was built keeping
future development in mind and thus, particular
focus was set on building expandable core
systems and interchangeable modular components
with accessible interfaces. An important
consideration when building an application is
following established software development principles.
Modern well-known software design principles
advocate single responsibility classes (such as
advocated by the SOLID principle [
          <xref ref-type="bibr" rid="ref75">75</xref>
          ]) and
urge not to repeat code. Developing an
application in line with these principles ensures a
modular system with accessible interfaces making
code reusable and testable. An added benefit
is the ease of updating individual components
of the application. For example, seeing the
location provider implemented in Arcane Shift
exposes the current coordinates of the player
through a public interface that other classes can
consume, the location provider can be updated
or completely changed at any point as long as it
exposes the same public interface that provides
coordinates. This becomes particularly
important when incorporating an increasing amount
of third-party software with unclear
development and maintenance roadmaps. Adhering to
established principles where sensible ensures
code is reusable and remains maintainable.
Using interfaces and favouring composition over
inheritance also allows for the easy addition of
new items such as diferent ores and a variety of
new buildings or new ship component upgrades.
This efectively allows for large updates of the
location-based game with new content, keeping
players engaged and motivated (with infrequent
large updates reported as preferable over
frequent small updates [
          <xref ref-type="bibr" rid="ref76">76</xref>
          ]), and as such arguably
increasing the likelihood of players contributing
to the various crowdsourcing tasks.
5.2.4. Spatial is special
A final key area worth considering is the
special nature of developing a globally available
spatial application. Firstly, visualising a
digital representation of a user’s surroundings as
the game world requires careful consideration
of the level of abstraction of the spatial
features. Adding too many details such as points
of interest or cluttered map labels can result
in cognitive overload [
          <xref ref-type="bibr" rid="ref77">77</xref>
          ] whilst reducing the
available screen real estate for virtual game
objects. Reducing map complexity could also
be in part responsible for the lack of altitude
information in many location-based games [
          <xref ref-type="bibr" rid="ref21">21</xref>
          ].
However, not adding enough spatial
information can confuse the user as to their in-game
location and the translation between real-world
and game-world coordinates. As such, striking
a balance between incorporating a minimalist
base map with enough spatial information to
ground the user in the game world is needed.
Secondly, seeing the potentially global scale of
location-based games, I recommend pursuing
a threefold content creation strategy: (1)
designed content, (2) user-generated content and
(3) procedurally generated content. Developing
the application to accommodate these three
distinct content generation strategies will allow
for specific areas of the game to be
purposefully designed especially in highly frequented
areas inviting the creation of custom content.
In addition, having user-generated content
allows for a dynamic and changing game world
and procedurally generated content ofers
globally available content. Finally, of particular
importance when designing applications geared
towards crowdsourcing spatial information is
considering the spatial and temporal
availability and repeatability of crowdsourcing tasks.
During the development of Arcane Shift, four
overarching categories of tasks emerged:
spatially and temporally repeatable, spatially but
not temporally repeatable, temporally but not
spatially repeatable and neither spatially nor
temporally repeatable. This allows for the
collection of detailed information about specific
areas over time as well as increasing spatial
coverage.
        </p>
      </sec>
      <sec id="sec-5-3">
        <title>5.3. Future agendas</title>
        <p>Within this paper, I have presented the design
decisions and the reasoning behind the
individual implementation steps in creating Arcane
Shift, a global gamified crowdsourcing platform.
In the following, I divulge three key lines of
research building on top of the implemented
platform. These aim to showcase the potential
of the application for sparking future research
in a wide variety of domains.</p>
        <p>
          Crowdsourcing and participatory approaches
have been identified as valuable complementary
approaches in policy and decision-making
contexts [
          <xref ref-type="bibr" rid="ref78 ref79 ref80">78, 79, 80</xref>
          ]. A first scientific avenue to
explore is thus the potential of using the
implemented location-based game to collect policy
and decision-making relevant spatial datasets.
Seeing the particular need for urban data to
pave the way for future urban living [
          <xref ref-type="bibr" rid="ref81 ref82">81, 82</xref>
          ]
in combination with the high spatial coverage
of mobile broadband internet in urban areas,
the implemented location-based game ofers
a particularly interesting platform to collect
a variety of urban information from the local
citizens themselves.
        </p>
        <p>
          Secondly, location-based games have shown
great potential in delivering certain educational
content [
          <xref ref-type="bibr" rid="ref83 ref84">83, 84</xref>
          ]. The possibility of
implementing chained quest steps as journeys opens the
door to investigating the platform’s ability to
deliver location-based educational content, which
may be of particular interest to educational
professionals seeking new ways to engage with
their students in a time of increased blended
and gamified learning [ 85]. The platform
offers the ability to create tailored educational
journeys, guiding learners through specific
realworld locations. Thus, testing educational
variables such as knowledge acquisition and learner
motivation could become an interesting future
investigation.
        </p>
        <p>Lastly, exergames (applications that
motivate physical activity), and location-based games
in general, have been found to increase
physical activity and in return afect users’ overall
health and well-being [86, 87]. This ofers a
third and important line of research for the
implemented application: questioning the
ability of the location-based game to motivate
increased physical activity. Motivating users for
physical activity in urban environments has
become an important dimension of urbanisation
research and entertaining motivational
incentives such as location-based games show great
potential.</p>
      </sec>
    </sec>
    <sec id="sec-6">
      <title>6. Limitations</title>
      <p>The implemented location-based game is
accompanied by various limitations, key of which
are listed here.</p>
      <p>Firstly, a large active player base and broad
spatial coverage of game content are needed to
make a virtual world feel alive and motivate
user engagement [88]. This limitation can be
somewhat mitigated by procedurally generated
content and non-player characters or large-scale
in-game events, conjuring the illusion of an
active and dynamic game world.</p>
      <p>
        A second key limitation is the requirement
of an adequate internet connection. This
excludes many rural areas which have generally
been underrepresented in location-based games
[
        <xref ref-type="bibr" rid="ref19">19</xref>
        ], potentially reinforcing the digital divide
[
        <xref ref-type="bibr" rid="ref5">5</xref>
        ]. However, an active internet connection
is required to allow for real-time updates as
needed for the game experience. As such, an
inevitable tradeof between real-time multiplayer
features and asynchronous ofline gaming must
be carefully considered. One possible approach
to address this limitation is to implement
specific ofline features accessible anywhere (e.g.
accepting a quest) which are not dependent
on real-time updates from other players or the
game world.
      </p>
      <p>One final major limitation of the implemented
location-based game is the dependence on
commercial products. The implemented platform
was built from scratch using state-of-the-art
technology and commercial products (e.g. Unity,
MapBox, FireStore) were used or integrated as
a means of minimising overall platform
maintenance. Access to these products may change
in the future, raising an important issue of
longevity. I try to address this issue by
implementing many of the core systems to be
modular and adapted if need be. For
example, the developed mapping system displaying
the base map can easily be modified to
display map tiles of other providers such as Open
Street Map.</p>
    </sec>
    <sec id="sec-7">
      <title>7. Conclusions</title>
      <p>Within this paper, I have presented the
development and implementation of a novel spatial
crowdsourcing platform in the form of a
globally available real-time location-based game.
The implemented application was not
developed to address a specific research question
in favour of serving as an underlying gamified
platform to host a variety of spatial
crowdsourcing initiatives globally. The design and
implementation of the application were grounded
in the Design Science Research Methodology
and the application was built using two
wellestablished frameworks. The location-based
game itself revolves around collecting and
forging resources, upgrading one’s spacecraft,
building structures, attacking enemy buildings and
completing location-based quests which are
crowdsourcing tasks. I further presented a
list of key design an implementation
recommendations, distilled from the lessons learnt
during the implementation of the application,
to contribute to the theoretical discussions on
implementing gamified spatial crowdsourcing
platforms or location-based games more
generally. Finally, by presenting three distinct future
research agendas, I hope to have highlighted
the intended impact and present real-world
scientific domains that would benefit from the
implemented platform. Arcane Shift hopes to
pave the way towards an era of entertaining
engagement in science and societally relevant
data collection.</p>
    </sec>
    <sec id="sec-8">
      <title>Acknowledgments</title>
      <p>I would like to extend my gratitude to the
internal test users for their amazing feedback. I
would also like to thank the anonymous
reviewers for their detailed feedback, considerably
improving the quality of this publication. This
project was funded by the Swiss National
Science Foundation (SNSF) under grant number
P500PT_214436.
games, Front. Educ. 6 (2021) 686633.
doi:10.3389/feduc.2021.686633.
[85] S. Hrastinski, What do we
mean by blended learning?,
TechTrends 63 (2019) 564–569.
doi:10.1007/s11528-019-00375-5.
[86] C. Mihale-Wilson, P. Felka, O. Hinz,
M. Spann, The influence of
locationbased games on traditional
entertainment products, Decis. Support Syst. 149
(2021) 113604. doi:10.1016/j.dss.2021.
113604.
[87] J. C. Davis, L. G. Killen, J. M. Green, H. S.</p>
      <p>Waldman, L. G. Renfroe, Exergaming for
physical activity: A systematic review, J
Am. Coll. Health (2022) 1–9. doi:10.1080/
07448481.2022.2103377.
[88] P. Fränti, N. Fazal, Design principles for
content creation in location-based games,
ACM Trans. Multimed. Comput.
Commun. Appl. (2023). doi:10.1145/3583689.</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          [1]
          <string-name>
            <given-names>L.</given-names>
            <surname>See</surname>
          </string-name>
          ,
          <string-name>
            <given-names>P.</given-names>
            <surname>Mooney</surname>
          </string-name>
          ,
          <string-name>
            <given-names>G.</given-names>
            <surname>Foody</surname>
          </string-name>
          ,
          <string-name>
            <given-names>L.</given-names>
            <surname>Bastin</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Comber</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Estima</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Fritz</surname>
          </string-name>
          ,
          <string-name>
            <given-names>N.</given-names>
            <surname>Kerle</surname>
          </string-name>
          ,
          <string-name>
            <given-names>B.</given-names>
            <surname>Jiang</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Laakso</surname>
          </string-name>
          , H.-Y. Liu, G. Milčinski,
          <string-name>
            <given-names>M.</given-names>
            <surname>Nikšič</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Painho</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Pődör</surname>
          </string-name>
          ,
          <string-name>
            <surname>A.-M. Olteanu-Raimond</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          <string-name>
            <surname>Rutzinger</surname>
          </string-name>
          ,
          <article-title>Crowdsourcing, citizen science or volunteered geographic information? the current state of crowdsourced geographic information</article-title>
          ,
          <source>ISPRS International Journal of Geo-Information</source>
          <volume>5</volume>
          (
          <year>2016</year>
          )
          <article-title>55</article-title>
          . doi:
          <volume>10</volume>
          . 3390/ijgi5050055.
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          [2]
          <string-name>
            <given-names>M.</given-names>
            <surname>Bubalo</surname>
          </string-name>
          ,
          <string-name>
            <surname>B. T. van Zanten</surname>
          </string-name>
          ,
          <string-name>
            <given-names>P. H.</given-names>
            <surname>Verburg</surname>
          </string-name>
          ,
          <article-title>Crowdsourcing geo-information on landscape perceptions and preferences: A review, Landsc</article-title>
          .
          <source>Urban Plan</source>
          .
          <volume>184</volume>
          (
          <year>2019</year>
          )
          <fpage>101</fpage>
          -
          <lpage>111</lpage>
          . doi:
          <volume>10</volume>
          .1016/ j.landurbplan.
          <year>2019</year>
          .
          <volume>01</volume>
          .001.
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          [3]
          <string-name>
            <given-names>Y.</given-names>
            <surname>Tong</surname>
          </string-name>
          ,
          <string-name>
            <given-names>Z.</given-names>
            <surname>Zhou</surname>
          </string-name>
          ,
          <string-name>
            <given-names>Y.</given-names>
            <surname>Zeng</surname>
          </string-name>
          ,
          <string-name>
            <given-names>L.</given-names>
            <surname>Chen</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C.</given-names>
            <surname>Shahabi</surname>
          </string-name>
          ,
          <article-title>Spatial crowdsourcing: a survey</article-title>
          ,
          <source>VLDB J</source>
          .
          <volume>29</volume>
          (
          <year>2020</year>
          )
          <fpage>217</fpage>
          -
          <lpage>250</lpage>
          . doi:
          <volume>10</volume>
          .1007/s00778-019-00568-7.
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          [4]
          <string-name>
            <given-names>R.</given-names>
            <surname>Lenart-Gansiniec</surname>
          </string-name>
          ,
          <string-name>
            <given-names>W.</given-names>
            <surname>Czakon</surname>
          </string-name>
          , Sułkowski,
          <string-name>
            <given-names>J.</given-names>
            <surname>Pocek</surname>
          </string-name>
          ,
          <article-title>Understanding crowdsourcing in science</article-title>
          ,
          <source>Rev. Manag. Sci</source>
          .
          <volume>17</volume>
          (
          <year>2023</year>
          )
          <fpage>2797</fpage>
          -
          <lpage>2830</lpage>
          . doi:
          <volume>10</volume>
          .1007/s11846-022-00602-z.
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          [5]
          <string-name>
            <given-names>M. F.</given-names>
            <surname>Goodchild</surname>
          </string-name>
          ,
          <article-title>Citizens as sensors: the world of volunteered geography</article-title>
          ,
          <source>GeoJournal</source>
          <volume>69</volume>
          (
          <year>2007</year>
          )
          <fpage>211</fpage>
          -
          <lpage>221</lpage>
          . doi:
          <volume>10</volume>
          .1007/ s10708-007-9111-y.
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          [6]
          <string-name>
            <given-names>L.</given-names>
            <surname>See</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Comber</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C.</given-names>
            <surname>Salk</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Fritz</surname>
          </string-name>
          , M. van der Velde, C. Perger,
          <string-name>
            <given-names>C.</given-names>
            <surname>Schill</surname>
          </string-name>
          ,
          <string-name>
            <surname>I. McCallum</surname>
          </string-name>
          ,
          <string-name>
            <given-names>F.</given-names>
            <surname>Kraxner</surname>
          </string-name>
          ,
          <string-name>
            <surname>M.</surname>
          </string-name>
          <article-title>Obersteiner, Comparing the quality of crowdsourced data contributed by expert and nonexperts</article-title>
          ,
          <source>PLoS One</source>
          <volume>8</volume>
          (
          <year>2013</year>
          )
          <fpage>1</fpage>
          -
          <lpage>11</lpage>
          . doi:
          <volume>10</volume>
          . 1371/journal.pone.
          <volume>0069958</volume>
          .
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          [7]
          <string-name>
            <given-names>A.</given-names>
            <surname>Ghermandi</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Sinclair</surname>
          </string-name>
          ,
          <article-title>Passive crowdsourcing of social media in environmental research: A systematic map</article-title>
          ,
          <source>Glob. Environ. Change</source>
          <volume>55</volume>
          (
          <year>2019</year>
          )
          <fpage>36</fpage>
          -
          <lpage>47</lpage>
          . doi:
          <volume>10</volume>
          . 1016/j.gloenvcha.
          <year>2019</year>
          .
          <volume>02</volume>
          .003.
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          [8]
          <string-name>
            <given-names>A.</given-names>
            <surname>Baruch</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>May</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Yu</surname>
          </string-name>
          ,
          <article-title>The motivations, enablers and barriers for voluntary participation in an online crowdsourcing platform</article-title>
          ,
          <source>Comput. Human Behav</source>
          .
          <volume>64</volume>
          (
          <year>2016</year>
          )
          <fpage>923</fpage>
          -
          <lpage>931</lpage>
          . doi:
          <volume>10</volume>
          .1016/j.chb.
          <year>2016</year>
          .
          <volume>07</volume>
          .039.
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          [9]
          <string-name>
            <given-names>B.</given-names>
            <surname>Morschheuser</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Hamari</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Koivisto</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Maedche</surname>
          </string-name>
          ,
          <article-title>Gamified crowdsourcing: Conceptualization, literature review, and future agenda</article-title>
          ,
          <source>International Journal of Human Computer Studies</source>
          <volume>106</volume>
          (
          <year>2017</year>
          )
          <fpage>26</fpage>
          -
          <lpage>43</lpage>
          . doi:
          <volume>10</volume>
          .1016/j.ijhcs.
          <year>2017</year>
          .
          <volume>04</volume>
          .005.
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          [10]
          <string-name>
            <given-names>L. F. S.</given-names>
            <surname>Pinto</surname>
          </string-name>
          , C. D. dos Santos, Motivations of crowdsourcing contributors,
          <source>Innovation &amp; Management Review</source>
          <volume>15</volume>
          (
          <year>2018</year>
          )
          <fpage>58</fpage>
          -
          <lpage>72</lpage>
          . doi:
          <volume>10</volume>
          .1108/inmr-02-2018-004.
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          [11]
          <string-name>
            <given-names>F. R.</given-names>
            <surname>Vianna</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Peinado</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A. R.</given-names>
            <surname>Graeml</surname>
          </string-name>
          ,
          <article-title>Crowdsourcing platforms: Objective, activities and motivation</article-title>
          ,
          <source>25th Americas Conference on Information Systems</source>
          ,
          <string-name>
            <surname>AMCIS</surname>
          </string-name>
          <year>2019</year>
          (
          <year>2019</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          [12]
          <string-name>
            <given-names>J.</given-names>
            <surname>Koivisto</surname>
          </string-name>
          ,
          <string-name>
            <surname>J. Hamari,</surname>
          </string-name>
          <article-title>The rise of motivational information systems: A review of gamification research</article-title>
          ,
          <source>Int. J. Inf. Manage</source>
          .
          <volume>45</volume>
          (
          <year>2019</year>
          )
          <fpage>191</fpage>
          -
          <lpage>210</lpage>
          . doi:
          <volume>10</volume>
          .1016/j. ijinfomgt.
          <year>2018</year>
          .
          <volume>10</volume>
          .013.
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          [13]
          <string-name>
            <surname>J. C. L. Bayas</surname>
            ,
            <given-names>L.</given-names>
          </string-name>
          <string-name>
            <surname>See</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          <string-name>
            <surname>Fritz</surname>
            ,
            <given-names>T.</given-names>
          </string-name>
          <string-name>
            <surname>Sturn</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          <string-name>
            <surname>Perger</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          <string-name>
            <surname>Dürauer</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          <string-name>
            <surname>Karner</surname>
            ,
            <given-names>I.</given-names>
          </string-name>
          <string-name>
            <surname>Moorthy</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          <string-name>
            <surname>Schepaschenko</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          <string-name>
            <surname>Domian</surname>
            ,
            <given-names>I. McCallum</given-names>
          </string-name>
          ,
          <article-title>Crowdsourcing insitu data on land cover and land use using gamification and mobile technology</article-title>
          ,
          <source>Remote Sensing</source>
          <volume>8</volume>
          (
          <year>2016</year>
          )
          <fpage>1</fpage>
          -
          <lpage>18</lpage>
          . doi:
          <volume>10</volume>
          .3390/ rs8110905.
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          [14]
          <string-name>
            <given-names>S.</given-names>
            <surname>Hantke</surname>
          </string-name>
          ,
          <string-name>
            <given-names>T.</given-names>
            <surname>Appel</surname>
          </string-name>
          ,
          <string-name>
            <given-names>B.</given-names>
            <surname>Schuller</surname>
          </string-name>
          ,
          <article-title>The inclusion of gamification solutions to enhance user enjoyment on crowdsourcing platforms</article-title>
          ,
          <source>2018 1st Asian Conference on Afective Computing and Intelligent Interaction, ACII Asia</source>
          <year>2018</year>
          (
          <year>2018</year>
          ). doi:
          <volume>10</volume>
          . 1109/ACIIAsia.
          <year>2018</year>
          .
          <volume>8470330</volume>
          .
        </mixed-citation>
      </ref>
      <ref id="ref15">
        <mixed-citation>
          [15]
          <string-name>
            <given-names>T.</given-names>
            <surname>Ivanjko</surname>
          </string-name>
          ,
          <article-title>Crowdsourcing image descriptions using gamification: A comparison between game-generated labels</article-title>
          and professional descriptors,
          <source>2019 42nd International Convention on Information and Communication Technology, Electronics and Microelectronics, MIPRO 2019 - Proceedings</source>
          (
          <year>2019</year>
          )
          <fpage>537</fpage>
          -
          <lpage>541</lpage>
          . doi:
          <volume>10</volume>
          .23919/ MIPRO.
          <year>2019</year>
          .
          <volume>8756841</volume>
          .
        </mixed-citation>
      </ref>
      <ref id="ref16">
        <mixed-citation>
          [16]
          <string-name>
            <given-names>M.</given-names>
            <surname>Baer</surname>
          </string-name>
          ,
          <string-name>
            <given-names>F. M.</given-names>
            <surname>Wartmann</surname>
          </string-name>
          , R. S. Purves,
          <article-title>StarBorn: Towards making in-situ land cover data generation fun with a location based game</article-title>
          ,
          <source>Trans. GIS</source>
          (
          <year>2019</year>
          )
          <fpage>1</fpage>
          -
          <lpage>21</lpage>
          . doi:
          <volume>10</volume>
          .1111/tgis.12543.
        </mixed-citation>
      </ref>
      <ref id="ref17">
        <mixed-citation>
          [17]
          <string-name>
            <given-names>B.</given-names>
            <surname>Morschheuser</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Hamari</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Maedche</surname>
          </string-name>
          ,
          <article-title>Cooperation or competition - when do people contribute more? a field experiment on gamification of crowdsourcing</article-title>
          ,
          <source>International Journal of Human Computer Studies</source>
          <volume>127</volume>
          (
          <year>2019</year>
          )
          <fpage>7</fpage>
          -
          <lpage>24</lpage>
          . doi:
          <volume>10</volume>
          .1016/j. ijhcs.
          <year>2018</year>
          .
          <volume>10</volume>
          .001.
        </mixed-citation>
      </ref>
      <ref id="ref18">
        <mixed-citation>
          [18]
          <string-name>
            <given-names>J.</given-names>
            <surname>Hamari</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Koivisto</surname>
          </string-name>
          ,
          <string-name>
            <given-names>H.</given-names>
            <surname>Sarsa</surname>
          </string-name>
          , Does gamification work?
          <article-title>- a literature review of empirical studies on gamification</article-title>
          ,
          <source>in: 2014 47th Hawaii International Conference on System Sciences</source>
          ,
          <year>2014</year>
          , pp.
          <fpage>3025</fpage>
          -
          <lpage>3034</lpage>
          . doi:
          <volume>10</volume>
          .1109/HICSS.
          <year>2014</year>
          .
          <volume>377</volume>
          .
        </mixed-citation>
      </ref>
      <ref id="ref19">
        <mixed-citation>
          [19]
          <string-name>
            <given-names>A.</given-names>
            <surname>Colley</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Thebault-Spieker</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A. Y.</given-names>
            <surname>Lin</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Degraen</surname>
          </string-name>
          ,
          <string-name>
            <given-names>B.</given-names>
            <surname>Fischman</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Häkkilä</surname>
          </string-name>
          ,
          <string-name>
            <given-names>K.</given-names>
            <surname>Kuehl</surname>
          </string-name>
          ,
          <string-name>
            <given-names>V.</given-names>
            <surname>Nisi</surname>
          </string-name>
          ,
          <string-name>
            <given-names>N. J.</given-names>
            <surname>Nunes</surname>
          </string-name>
          ,
          <string-name>
            <given-names>N.</given-names>
            <surname>Wenig</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Wenig</surname>
          </string-name>
          ,
          <string-name>
            <given-names>B.</given-names>
            <surname>Hecht</surname>
          </string-name>
          ,
          <string-name>
            <surname>J. Schöning,</surname>
          </string-name>
          <article-title>The geography of pokémon GO: Beneficial and problematic efects on places and movement</article-title>
          , arXiv (
          <year>2017</year>
          )
          <fpage>1179</fpage>
          -
          <lpage>1192</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref20">
        <mixed-citation>
          [20]
          <string-name>
            <given-names>S.</given-names>
            <surname>Laato</surname>
          </string-name>
          ,
          <string-name>
            <surname>A. K. M. N. Islam</surname>
            ,
            <given-names>T. H.</given-names>
          </string-name>
          <string-name>
            <surname>Laine</surname>
          </string-name>
          ,
          <article-title>Did location-based games motivate players to socialize during COVID-19?</article-title>
          , Telemat. Inform.
          <volume>54</volume>
          (
          <year>2020</year>
          )
          <article-title>101458</article-title>
          . doi:
          <volume>10</volume>
          .1016/j. tele.
          <year>2020</year>
          .
          <volume>101458</volume>
          .
        </mixed-citation>
      </ref>
      <ref id="ref21">
        <mixed-citation>
          [21]
          <string-name>
            <given-names>M.</given-names>
            <surname>Baer</surname>
          </string-name>
          ,
          <string-name>
            <given-names>T.</given-names>
            <surname>Tregel</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Laato</surname>
          </string-name>
          ,
          <string-name>
            <given-names>H.</given-names>
            <surname>Söbke</surname>
          </string-name>
          ,
          <article-title>Virtually (re)constructed reality: the representation of physical space in commercial location-based games</article-title>
          ,
          <source>in: Proceedings of the 25th International Academic Mindtrek Conference</source>
          , ACM, New York, NY, USA,
          <year>2022</year>
          . doi:
          <volume>10</volume>
          .1145/3569219. 3569339.
        </mixed-citation>
      </ref>
      <ref id="ref22">
        <mixed-citation>
          [22]
          <string-name>
            <given-names>S.</given-names>
            <surname>Laato</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S. M.</given-names>
            <surname>Hyrynsalmi</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Paloheimo</surname>
          </string-name>
          ,
          <article-title>Online multiplayer games for crowdsourcing the development of digital assets: The case of ingress</article-title>
          ,
          <source>in: Lecture Notes in Business Information Processing, Lecture notes in business information processing</source>
          , Springer International Publishing, Cham,
          <year>2019</year>
          , pp.
          <fpage>387</fpage>
          -
          <lpage>401</lpage>
          . doi:
          <volume>10</volume>
          .1007/978-3-
          <fpage>030</fpage>
          -33742-1\_
          <fpage>31</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref23">
        <mixed-citation>
          [23]
          <string-name>
            <given-names>S.</given-names>
            <surname>Laato</surname>
          </string-name>
          , T. Tregel,
          <article-title>Into the unown: Improving location-based gamified crowdsourcing solutions for geo data gathering</article-title>
          ,
          <source>Entertain. Comput</source>
          .
          <volume>46</volume>
          (
          <year>2023</year>
          )
          <article-title>100575</article-title>
          . doi:
          <volume>10</volume>
          .1016/j.entcom.
          <year>2023</year>
          .
          <volume>100575</volume>
          .
        </mixed-citation>
      </ref>
      <ref id="ref24">
        <mixed-citation>
          [24]
          <string-name>
            <given-names>D.</given-names>
            <surname>Leorke</surname>
          </string-name>
          ,
          <article-title>A definition and brief history of location-based games (</article-title>
          <year>2001</year>
          -08),
          <source>in: Location-Based Gaming: Play in Public Space</source>
          , Springer Singapore, Singapore,
          <year>2019</year>
          , pp.
          <fpage>17</fpage>
          -
          <lpage>44</lpage>
          . doi:
          <volume>10</volume>
          .1007/
          <fpage>978</fpage>
          -981-13-0683-9\_2.
        </mixed-citation>
      </ref>
      <ref id="ref25">
        <mixed-citation>
          [25]
          <string-name>
            <surname>De Souza E Silva</surname>
          </string-name>
          , L. Hjorth,
          <article-title>Playful urban spaces: Historical approach mobile games</article-title>
          ,
          <source>Simulation &amp; Gaming</source>
          <volume>40</volume>
          (
          <year>2009</year>
          )
          <fpage>602</fpage>
          -
          <lpage>625</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref26">
        <mixed-citation>
          [26]
          <string-name>
            <given-names>L.</given-names>
            <surname>Juhász</surname>
          </string-name>
          ,
          <string-name>
            <given-names>H. H.</given-names>
            <surname>Hochmair</surname>
          </string-name>
          , Where to catch 'em all?
          <article-title>-a geographic analysis of pokémon go locations</article-title>
          ,
          <source>Geo Spat. Inf. Sci</source>
          .
          <volume>20</volume>
          (
          <year>2017</year>
          )
          <fpage>241</fpage>
          -
          <lpage>251</lpage>
          . doi:
          <volume>10</volume>
          .1080/ 10095020.
          <year>2017</year>
          .
          <volume>1368200</volume>
          .
        </mixed-citation>
      </ref>
      <ref id="ref27">
        <mixed-citation>
          [27]
          <string-name>
            <given-names>I.</given-names>
            <surname>Celino</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Cerizza</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Contessa</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Corubolo</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Dellaglio</surname>
          </string-name>
          ,
          <string-name>
            <given-names>E. D.</given-names>
            <surname>Valle</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Fumeo</surname>
          </string-name>
          ,
          <article-title>Urbanopoly - a social and location-based game with a purpose to crowdsource your urban data</article-title>
          ,
          <source>in: Proceedings - 2012 ASE/IEEE International Conference on Privacy, Security, Risk and Trust and 2012 ASE/IEEE International Conference on Social Computing, SocialCom/PASSAT</source>
          <year>2012</year>
          ,
          <year>2012</year>
          , pp.
          <fpage>910</fpage>
          -
          <lpage>913</lpage>
          . doi:
          <volume>10</volume>
          .1109/ SocialCom-PASSAT.
          <year>2012</year>
          .
          <volume>138</volume>
          .
        </mixed-citation>
      </ref>
      <ref id="ref28">
        <mixed-citation>
          [28]
          <string-name>
            <given-names>A.</given-names>
            <surname>Abraham</surname>
          </string-name>
          ,
          <string-name>
            <given-names>K.</given-names>
            <surname>Sommerhalder</surname>
          </string-name>
          , T. Abel,
          <article-title>Landscape and well-being: A scoping study on the health-promoting impact of outdoor environments</article-title>
          ,
          <source>Int. J. Public Health</source>
          <volume>55</volume>
          (
          <year>2010</year>
          )
          <fpage>59</fpage>
          -
          <lpage>69</lpage>
          . doi:
          <volume>10</volume>
          .1007/ s00038-009-0069-z.
        </mixed-citation>
      </ref>
      <ref id="ref29">
        <mixed-citation>
          [29]
          <string-name>
            <given-names>C. W.</given-names>
            <surname>Thompson</surname>
          </string-name>
          ,
          <article-title>Linking landscape and health: The recurring theme</article-title>
          ,
          <source>Landsc. Urban Plan</source>
          .
          <volume>99</volume>
          (
          <year>2011</year>
          )
          <fpage>187</fpage>
          -
          <lpage>195</lpage>
          . doi:
          <volume>10</volume>
          . 1016/j.landurbplan.
          <year>2010</year>
          .
          <volume>10</volume>
          .006.
        </mixed-citation>
      </ref>
      <ref id="ref30">
        <mixed-citation>
          [30]
          <string-name>
            <given-names>L.</given-names>
            <surname>Menatti</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A. C.</given-names>
            <surname>Da Rocha</surname>
          </string-name>
          ,
          <article-title>Landscape and health: Connecting psychology, aesthetics, and philosophy through the concept of afordance</article-title>
          ,
          <source>Front. Psychol</source>
          .
          <volume>7</volume>
          (
          <issue>2016</issue>
          )
          <fpage>1</fpage>
          -
          <lpage>17</lpage>
          . doi:
          <volume>10</volume>
          .3389/fpsyg.
          <year>2016</year>
          .
          <volume>00571</volume>
          .
        </mixed-citation>
      </ref>
      <ref id="ref31">
        <mixed-citation>
          [31]
          <string-name>
            <given-names>C.</given-names>
            <surname>Llinares</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J. L.</given-names>
            <surname>Higuera-Trujillo</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Montañana</surname>
          </string-name>
          ,
          <string-name>
            <given-names>N.</given-names>
            <surname>Castilla</surname>
          </string-name>
          ,
          <article-title>Improving the pedestrian's perceptions of safety on street crossings. psychological and neurophysiological efects of trafic lanes, artificial lighting, and vegetation</article-title>
          ,
          <source>Int. J. Environ. Res. Public Health</source>
          <volume>17</volume>
          (
          <year>2020</year>
          )
          <article-title>8576</article-title>
          . doi:
          <volume>10</volume>
          .3390/ijerph17228576.
        </mixed-citation>
      </ref>
      <ref id="ref32">
        <mixed-citation>
          [32]
          <string-name>
            <given-names>E.</given-names>
            <surname>Zeng</surname>
          </string-name>
          ,
          <string-name>
            <given-names>Y.</given-names>
            <surname>Dong</surname>
          </string-name>
          ,
          <string-name>
            <given-names>L.</given-names>
            <surname>Yan</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Lin</surname>
          </string-name>
          ,
          <article-title>Perceived safety in the neighborhood: Exploring the role of built environment, social factors, physical activity and multiple pathways of influence</article-title>
          ,
          <source>Buildings</source>
          <volume>13</volume>
          (
          <year>2022</year>
          )
          <article-title>2</article-title>
          . doi:
          <volume>10</volume>
          .3390/buildings13010002.
        </mixed-citation>
      </ref>
      <ref id="ref33">
        <mixed-citation>
          [33]
          <string-name>
            <given-names>F. M.</given-names>
            <surname>Wartmann</surname>
          </string-name>
          ,
          <string-name>
            <given-names>K. F.</given-names>
            <surname>Tieskens</surname>
          </string-name>
          ,
          <string-name>
            <surname>B. T. van Zanten</surname>
          </string-name>
          ,
          <string-name>
            <given-names>P. H.</given-names>
            <surname>Verburg</surname>
          </string-name>
          ,
          <article-title>Exploring tranquillity experienced in landscapes based on social media</article-title>
          ,
          <source>Appl. Geogr</source>
          .
          <volume>113</volume>
          (
          <year>2019</year>
          )
          <article-title>102112</article-title>
          . doi:
          <volume>10</volume>
          .1016/j. apgeog.
          <year>2019</year>
          .
          <volume>102112</volume>
          .
        </mixed-citation>
      </ref>
      <ref id="ref34">
        <mixed-citation>
          [34]
          <string-name>
            <given-names>N.</given-names>
            <surname>Fagerholm</surname>
          </string-name>
          ,
          <string-name>
            <given-names>B.</given-names>
            <surname>Martín-López</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Torralba</surname>
          </string-name>
          ,
          <string-name>
            <given-names>E.</given-names>
            <surname>Oteros-Rozas</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A. M.</given-names>
            <surname>Lechner</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C.</given-names>
            <surname>Bieling</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A. Stahl</given-names>
            <surname>Olafsson</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C.</given-names>
            <surname>Albert</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C. M.</given-names>
            <surname>Raymond</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Garcia-Martin</surname>
          </string-name>
          ,
          <string-name>
            <given-names>N.</given-names>
            <surname>Gulsrud</surname>
          </string-name>
          , T. Plieninger,
          <article-title>Perceived contributions of multifunctional landscapes to human well-being: Evidence from 13 european sites</article-title>
          ,
          <source>People and Nature</source>
          <volume>2</volume>
          (
          <year>2020</year>
          )
          <fpage>217</fpage>
          -
          <lpage>234</lpage>
          . doi:
          <volume>10</volume>
          .1002/pan3.
          <fpage>10067</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref35">
        <mixed-citation>
          [35]
          <string-name>
            <given-names>G.</given-names>
            <surname>Watts</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R.</given-names>
            <surname>Pheasant</surname>
          </string-name>
          ,
          <string-name>
            <given-names>K.</given-names>
            <surname>Horoshenkov</surname>
          </string-name>
          ,
          <article-title>Tranquil spaces in a metropolitan area</article-title>
          ,
          <source>in: Proceedings of the 20th International Congress on Acoustics, Sydney</source>
          , Australia, acoustics.asn.au,
          <year>2010</year>
          , pp.
          <fpage>23</fpage>
          -
          <lpage>27</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref36">
        <mixed-citation>
          [36]
          <string-name>
            <given-names>G.</given-names>
            <surname>Waters</surname>
          </string-name>
          ,
          <string-name>
            <given-names>B.</given-names>
            <surname>Warren</surname>
          </string-name>
          ,
          <string-name>
            <given-names>E.</given-names>
            <surname>Ratclife</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Godefroy</surname>
          </string-name>
          ,
          <article-title>Tranquil city: identifying opportunities for urban tranquillity to promote healthy lifestyles</article-title>
          ,
          <source>Cities &amp; Health</source>
          <volume>5</volume>
          (
          <year>2021</year>
          )
          <fpage>138</fpage>
          -
          <lpage>144</lpage>
          . doi:
          <volume>10</volume>
          .1080/23748834.
          <year>2019</year>
          .
          <volume>1617918</volume>
          .
        </mixed-citation>
      </ref>
      <ref id="ref37">
        <mixed-citation>
          [37]
          <string-name>
            <given-names>C.</given-names>
            <surname>Rissel</surname>
          </string-name>
          ,
          <string-name>
            <given-names>N.</given-names>
            <surname>Curac</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Greenaway</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Bauman</surname>
          </string-name>
          ,
          <article-title>Physical activity associated with public transport use-a review and modelling of potential benefits</article-title>
          ,
          <source>Int. J. Environ. Res. Public Health</source>
          <volume>9</volume>
          (
          <year>2012</year>
          )
          <fpage>2454</fpage>
          -
          <lpage>2478</lpage>
          . doi:
          <volume>10</volume>
          .3390/ijerph9072454.
        </mixed-citation>
      </ref>
      <ref id="ref38">
        <mixed-citation>
          [38]
          <string-name>
            <given-names>I.</given-names>
            <surname>Batur</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S. A.</given-names>
            <surname>Markolf</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M. V.</given-names>
            <surname>Chester</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Middel</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Hondula</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Vanos</surname>
          </string-name>
          ,
          <article-title>Streetlevel heat and air pollution exposure informed by mobile sensing, Transp</article-title>
          . Res.
          <string-name>
            <given-names>D</given-names>
            <surname>Transp</surname>
          </string-name>
          .
          <year>Environ</year>
          .
          <volume>113</volume>
          (
          <year>2022</year>
          )
          <article-title>103535</article-title>
          . doi:
          <volume>10</volume>
          .1016/j.trd.
          <year>2022</year>
          .
          <volume>103535</volume>
          .
        </mixed-citation>
      </ref>
      <ref id="ref39">
        <mixed-citation>
          [39]
          <string-name>
            <given-names>K.</given-names>
            <surname>Fuladlu</surname>
          </string-name>
          ,
          <string-name>
            <given-names>H.</given-names>
            <surname>Altan</surname>
          </string-name>
          ,
          <article-title>Examining land surface temperature and relations with the major air pollutants: A remote sensing research in case of tehran</article-title>
          , Urban Clim.
          <volume>39</volume>
          (
          <year>2021</year>
          )
          <article-title>100958</article-title>
          . doi:
          <volume>10</volume>
          .1016/j.uclim.
          <year>2021</year>
          .
          <volume>100958</volume>
          .
        </mixed-citation>
      </ref>
      <ref id="ref40">
        <mixed-citation>
          [40]
          <string-name>
            <given-names>H.</given-names>
            <surname>Wang</surname>
          </string-name>
          ,
          <string-name>
            <given-names>Y.</given-names>
            <surname>Yang</surname>
          </string-name>
          ,
          <article-title>Neighbourhood walkability: A review and bibliometric analysis</article-title>
          ,
          <source>Cities</source>
          <volume>93</volume>
          (
          <year>2019</year>
          )
          <fpage>43</fpage>
          -
          <lpage>61</lpage>
          . doi:
          <volume>10</volume>
          .1016/ j.cities.
          <year>2019</year>
          .
          <volume>04</volume>
          .015.
        </mixed-citation>
      </ref>
      <ref id="ref41">
        <mixed-citation>
          [41]
          <string-name>
            <given-names>C.</given-names>
            <surname>Legacy</surname>
          </string-name>
          ,
          <article-title>Investigating the knowledge interface between stakeholder engagement and plan-making</article-title>
          ,
          <source>Environ. Plan. A</source>
          <volume>42</volume>
          (
          <year>2010</year>
          )
          <fpage>2705</fpage>
          -
          <lpage>2720</lpage>
          . doi:
          <volume>10</volume>
          .1068/a43164.
        </mixed-citation>
      </ref>
      <ref id="ref42">
        <mixed-citation>
          [42]
          <string-name>
            <given-names>R. G.</given-names>
            <surname>Hollands</surname>
          </string-name>
          ,
          <article-title>Critical interventions into the corporate smart city</article-title>
          ,
          <source>Camb. J. Reg. Econ. Soc</source>
          .
          <volume>8</volume>
          (
          <year>2015</year>
          )
          <fpage>61</fpage>
          -
          <lpage>77</lpage>
          . doi:
          <volume>10</volume>
          .1093/ cjres/rsu011.
        </mixed-citation>
      </ref>
      <ref id="ref43">
        <mixed-citation>
          [43]
          <string-name>
            <given-names>G.</given-names>
            <surname>Brown</surname>
          </string-name>
          , M. Kyttä,
          <article-title>Key issues and research priorities for public participation GIS (PPGIS): A synthesis based on empirical research</article-title>
          , Appl. Geogr.
          <volume>46</volume>
          (
          <year>2014</year>
          )
          <fpage>122</fpage>
          -
          <lpage>136</lpage>
          . doi:
          <volume>10</volume>
          .1016/j.apgeog.
          <year>2013</year>
          .
          <volume>11</volume>
          .004.
        </mixed-citation>
      </ref>
      <ref id="ref44">
        <mixed-citation>
          [44]
          <string-name>
            <given-names>E.</given-names>
            <surname>Estellés-Arolas</surname>
          </string-name>
          ,
          <string-name>
            <given-names>F.</given-names>
            <surname>González-LadrónDe-Guevara</surname>
          </string-name>
          ,
          <article-title>Towards an integrated crowdsourcing definition</article-title>
          ,
          <source>J. Inf. Sci. Eng</source>
          .
          <volume>38</volume>
          (
          <year>2012</year>
          )
          <fpage>189</fpage>
          -
          <lpage>200</lpage>
          . doi:
          <volume>10</volume>
          .1177/ 0165551512437638.
        </mixed-citation>
      </ref>
      <ref id="ref45">
        <mixed-citation>
          [45]
          <string-name>
            <given-names>D.</given-names>
            <surname>Quercia</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R.</given-names>
            <surname>Schifanella</surname>
          </string-name>
          ,
          <string-name>
            <given-names>L. M.</given-names>
            <surname>Aiello</surname>
          </string-name>
          ,
          <string-name>
            <given-names>K.</given-names>
            <surname>McLean</surname>
          </string-name>
          ,
          <article-title>Smelly maps: The digital life of urban smellscapes</article-title>
          ,
          <source>Association for the Advancement of Artificial Intelligence</source>
          <volume>12</volume>
          (
          <year>2015</year>
          ). arXiv:
          <volume>1505</volume>
          .
          <fpage>06851</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref46">
        <mixed-citation>
          [46]
          <string-name>
            <given-names>L. M.</given-names>
            <surname>Aiello</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R.</given-names>
            <surname>Schifanella</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Quercia</surname>
          </string-name>
          ,
          <string-name>
            <given-names>F.</given-names>
            <surname>Aletta</surname>
          </string-name>
          ,
          <article-title>Chatty maps : constructing sound maps of urban areas from social media data</article-title>
          ,
          <source>R. Soc. open sci. 3</source>
          (
          <year>2016</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref47">
        <mixed-citation>
          [47]
          <string-name>
            <given-names>G.</given-names>
            <surname>Lamprianidis</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Skoutas</surname>
          </string-name>
          ,
          <string-name>
            <given-names>G.</given-names>
            <surname>Papatheodorou</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Pfoser</surname>
          </string-name>
          ,
          <article-title>Extraction, integration and analysis of crowdsourced points of interest from multiple web sources</article-title>
          ,
          <source>in: Proceedings of the 3rd ACM SIGSPATIAL International Workshop on Crowdsourced and Volunteered Geographic Information</source>
          ,
          <string-name>
            <surname>ACM</surname>
          </string-name>
          , New York, NY, USA,
          <year>2014</year>
          . doi:
          <volume>10</volume>
          .1145/2676440. 2676445.
        </mixed-citation>
      </ref>
      <ref id="ref48">
        <mixed-citation>
          [48]
          <string-name>
            <given-names>S. T.</given-names>
            <surname>March</surname>
          </string-name>
          ,
          <string-name>
            <given-names>G. F.</given-names>
            <surname>Smith</surname>
          </string-name>
          ,
          <source>Design and natural science research on information technology, Decis. Support Syst</source>
          .
          <volume>15</volume>
          (
          <year>1995</year>
          )
          <fpage>251</fpage>
          -
          <lpage>266</lpage>
          . doi:
          <volume>10</volume>
          .1016/
          <fpage>0167</fpage>
          -
          <lpage>9236</lpage>
          (
          <issue>94</issue>
          )
          <fpage>00041</fpage>
          -
          <lpage>2</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref49">
        <mixed-citation>
          [49]
          <string-name>
            <given-names>K.</given-names>
            <surname>Pefers</surname>
          </string-name>
          ,
          <string-name>
            <given-names>T.</given-names>
            <surname>Tuunanen</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M. A.</given-names>
            <surname>Rothenberger</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Chatterjee</surname>
          </string-name>
          ,
          <article-title>A design science research methodology for information systems research</article-title>
          ,
          <source>J. Manag. Inf. Syst</source>
          .
          <volume>24</volume>
          (
          <year>2007</year>
          )
          <fpage>45</fpage>
          -
          <lpage>77</lpage>
          . doi:
          <volume>10</volume>
          .2753/ mis0742-
          <fpage>1222240302</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref50">
        <mixed-citation>
          [50]
          <string-name>
            <given-names>R.</given-names>
            <surname>Winter</surname>
          </string-name>
          , Design science research in europe,
          <source>Eur. J. Inf. Syst</source>
          .
          <volume>17</volume>
          (
          <year>2008</year>
          )
          <fpage>470</fpage>
          -
          <lpage>475</lpage>
          . doi:
          <volume>10</volume>
          .1057/ejis.
          <year>2008</year>
          .
          <volume>44</volume>
          .
        </mixed-citation>
      </ref>
      <ref id="ref51">
        <mixed-citation>
          [51]
          <string-name>
            <surname>J. vom Brocke</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          <string-name>
            <surname>Hevner</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          <string-name>
            <surname>Maedche</surname>
          </string-name>
          , Introduction to design
          <source>science research</source>
          , in: Progress in
          <string-name>
            <surname>IS</surname>
          </string-name>
          , Springer International Publishing, Cham,
          <year>2020</year>
          , pp.
          <fpage>1</fpage>
          -
          <lpage>13</lpage>
          . doi:
          <volume>10</volume>
          . 1007/978-3-
          <fpage>030</fpage>
          -46781-4\_1.
        </mixed-citation>
      </ref>
      <ref id="ref52">
        <mixed-citation>
          [52]
          <string-name>
            <given-names>R.</given-names>
            <surname>Hunicke</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Leblanc</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R.</given-names>
            <surname>Zubek</surname>
          </string-name>
          ,
          <article-title>MDA: A formal approach to game design and game research</article-title>
          ,
          <source>in: Proceedings of the Challenges in Games AI Workshop</source>
          , AAAI Press, Menlo Park, CA,
          <year>2004</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref53">
        <mixed-citation>
          [53]
          <string-name>
            <given-names>R.</given-names>
            <surname>Martella</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C.</given-names>
            <surname>Kray</surname>
          </string-name>
          ,
          <string-name>
            <given-names>E.</given-names>
            <surname>Clementini</surname>
          </string-name>
          ,
          <article-title>A gamification framework for volunteered geographic information</article-title>
          , in: F. Bacao,
          <string-name>
            <given-names>M. Y.</given-names>
            <surname>Santos</surname>
          </string-name>
          , M. Painho (Eds.), AGILE 2015:
          <article-title>Geographic Information Science as an Enabler of Smarter Cities and Communities</article-title>
          , volume
          <volume>217</volume>
          , Springer International Publishing, Cham,
          <year>2015</year>
          , pp.
          <fpage>73</fpage>
          -
          <lpage>89</lpage>
          . doi:
          <volume>10</volume>
          .1007/978-3-
          <fpage>319</fpage>
          -16787-9\_5.
        </mixed-citation>
      </ref>
      <ref id="ref54">
        <mixed-citation>
          [54]
          <string-name>
            <given-names>G. P.</given-names>
            <surname>Kusuma</surname>
          </string-name>
          ,
          <string-name>
            <given-names>E. K.</given-names>
            <surname>Wigati</surname>
          </string-name>
          ,
          <string-name>
            <given-names>Y.</given-names>
            <surname>Utomo</surname>
          </string-name>
          ,
          <string-name>
            <given-names>L. K.</given-names>
            <surname>Putera Suryapranata</surname>
          </string-name>
          ,
          <article-title>Analysis of gamification models in education using MDA framework</article-title>
          ,
          <source>Procedia Comput. Sci</source>
          .
          <volume>135</volume>
          (
          <year>2018</year>
          )
          <fpage>385</fpage>
          -
          <lpage>392</lpage>
          . doi:
          <volume>10</volume>
          .1016/j. procs.
          <year>2018</year>
          .
          <volume>08</volume>
          .187.
        </mixed-citation>
      </ref>
      <ref id="ref55">
        <mixed-citation>
          [55]
          <string-name>
            <given-names>V.</given-names>
            <surname>Adarsh</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Nekrasov</surname>
          </string-name>
          , U. Paul,
          <string-name>
            <given-names>T.</given-names>
            <surname>Mangla</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Gupta</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Vigil-Hayes</surname>
          </string-name>
          ,
          <string-name>
            <given-names>E.</given-names>
            <surname>Zegura</surname>
          </string-name>
          , E. Belding,
          <article-title>Coverage is not binary: Quantifying mobile broadband quality in urban, rural, and tribal contexts</article-title>
          ,
          <source>in: 2021 International Conference on Computer Communications and Networks (ICCCN)</source>
          , IEEE,
          <year>2021</year>
          . doi:
          <volume>10</volume>
          . 1109/icccn52240.
          <year>2021</year>
          .
          <volume>9522152</volume>
          .
        </mixed-citation>
      </ref>
      <ref id="ref56">
        <mixed-citation>
          [56]
          <string-name>
            <surname>P. M. Duvall</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          <string-name>
            <surname>Matyas</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          <string-name>
            <surname>Glover</surname>
          </string-name>
          , Continuous Integration:
          <article-title>Improving Software Quality</article-title>
          and
          <string-name>
            <given-names>Reducing</given-names>
            <surname>Risk</surname>
          </string-name>
          , Pearson Education, Philadelphia, PA,
          <year>2007</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref57">
        <mixed-citation>
          [57]
          <string-name>
            <given-names>M.</given-names>
            <surname>Shahin</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M. Ali</given-names>
            <surname>Babar</surname>
          </string-name>
          ,
          <string-name>
            <given-names>L.</given-names>
            <surname>Zhu</surname>
          </string-name>
          ,
          <article-title>Continuous integration, delivery and deployment: A systematic review on approaches, tools, challenges and practices</article-title>
          ,
          <source>IEEE Access 5</source>
          (
          <year>2017</year>
          )
          <fpage>3909</fpage>
          -
          <lpage>3943</lpage>
          . doi:
          <volume>10</volume>
          .1109/access.
          <year>2017</year>
          .
          <volume>2685629</volume>
          .
        </mixed-citation>
      </ref>
      <ref id="ref58">
        <mixed-citation>
          [58]
          <string-name>
            <given-names>A.</given-names>
            <surname>Henry</surname>
          </string-name>
          ,
          <article-title>L2 motivation and multilingual identities, Mod</article-title>
          . Lang. J.
          <volume>101</volume>
          (
          <year>2017</year>
          )
          <fpage>548</fpage>
          -
          <lpage>565</lpage>
          . doi:
          <volume>10</volume>
          .1111/modl.12412.
        </mixed-citation>
      </ref>
      <ref id="ref59">
        <mixed-citation>
          [59]
          <string-name>
            <given-names>A.</given-names>
            <surname>Cardinal</surname>
          </string-name>
          ,
          <string-name>
            <given-names>L.</given-names>
            <surname>Gonzales</surname>
          </string-name>
          ,
          <string-name>
            <given-names>E. J.</given-names>
            <surname>Rose</surname>
          </string-name>
          ,
          <article-title>Language as participation: Multilingual user experience design</article-title>
          ,
          <source>in: Proceedings of the 38th ACM International Conference on Design of Communication</source>
          , ACM, New York, NY, USA,
          <year>2020</year>
          . doi:
          <volume>10</volume>
          .1145/ 3380851.3416763.
        </mixed-citation>
      </ref>
      <ref id="ref60">
        <mixed-citation>
          [60]
          <string-name>
            <surname>D. M. Da Rosa</surname>
            ,
            <given-names>M. L. L.</given-names>
          </string-name>
          <string-name>
            <surname>Souto</surname>
            ,
            <given-names>M. S.</given-names>
          </string-name>
          <string-name>
            <surname>Silveira</surname>
          </string-name>
          ,
          <article-title>Designing interfaces for multilingual users: a pattern language</article-title>
          ,
          <source>in: Proceedings of the 27th European Conference on Pattern Languages of Programs</source>
          , ACM, New York, NY, USA,
          <year>2022</year>
          . doi:
          <volume>10</volume>
          .1145/3551902.3551964.
        </mixed-citation>
      </ref>
      <ref id="ref61">
        <mixed-citation>
          [61]
          <string-name>
            <given-names>F.</given-names>
            <surname>Kreuter</surname>
          </string-name>
          ,
          <string-name>
            <given-names>G.-C.</given-names>
            <surname>Haas</surname>
          </string-name>
          ,
          <string-name>
            <given-names>F.</given-names>
            <surname>Keusch</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Bähr</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Trappmann</surname>
          </string-name>
          ,
          <article-title>Collecting survey and smartphone sensor data with an app: Opportunities and challenges around privacy and informed consent</article-title>
          ,
          <source>Soc. Sci. Comput. Rev</source>
          .
          <volume>38</volume>
          (
          <year>2020</year>
          )
          <fpage>533</fpage>
          -
          <lpage>549</lpage>
          . doi:
          <volume>10</volume>
          .1177/0894439318816389.
        </mixed-citation>
      </ref>
      <ref id="ref62">
        <mixed-citation>
          [62]
          <string-name>
            <given-names>S.</given-names>
            <surname>Nicholson</surname>
          </string-name>
          ,
          <article-title>A RECIPE for meaningful gamification</article-title>
          ,
          <source>in: Gamification in Education and Business</source>
          , Springer International Publishing, Cham,
          <year>2015</year>
          , pp.
          <fpage>1</fpage>
          -
          <lpage>20</lpage>
          . doi:
          <volume>10</volume>
          .1007/978-3-
          <fpage>319</fpage>
          -10208-5\_1.
        </mixed-citation>
      </ref>
      <ref id="ref63">
        <mixed-citation>
          [63]
          <string-name>
            <given-names>H.</given-names>
            <surname>Wang</surname>
          </string-name>
          , C.-T. Sun,
          <article-title>Game reward systems : Gaming experiences and social meanings</article-title>
          ,
          <source>in: Proceedings of DiGRA 2011 Conference: Think Design Play.</source>
          ,
          <year>2011</year>
          , pp.
          <fpage>1</fpage>
          -
          <lpage>15</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref64">
        <mixed-citation>
          [64]
          <string-name>
            <given-names>J.</given-names>
            <surname>Brich</surname>
          </string-name>
          ,
          <article-title>Motivational game design factors in player-game adaptivity</article-title>
          ,
          <source>in: CHI PLAY '17 Extended Abstracts: Extended Abstracts Publication of the Annual Symposium on Computer-Human Interaction in Play</source>
          ,
          <year>2017</year>
          , pp.
          <fpage>683</fpage>
          -
          <lpage>686</lpage>
          . doi:
          <volume>10</volume>
          .1145/ 3130859.3133222.
        </mixed-citation>
      </ref>
      <ref id="ref65">
        <mixed-citation>
          [65]
          <string-name>
            <given-names>S.</given-names>
            <surname>Fritz</surname>
          </string-name>
          ,
          <string-name>
            <given-names>L.</given-names>
            <surname>See</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Brovelli</surname>
          </string-name>
          ,
          <article-title>Motivating and sustaining participation in VGI</article-title>
          , in: S. Fritz,
          <string-name>
            <given-names>L.</given-names>
            <surname>See</surname>
          </string-name>
          , G. Foody,
          <string-name>
            <given-names>P.</given-names>
            <surname>Mooney</surname>
          </string-name>
          ,
          <string-name>
            <surname>A.- M. Olteanu-Raimond</surname>
            ,
            <given-names>C. C.</given-names>
          </string-name>
          <string-name>
            <surname>Fonte</surname>
          </string-name>
          , V. Antoniou (Eds.),
          <article-title>Mapping and the Citizen Sensor</article-title>
          , Ubiquity Press,
          <year>2017</year>
          , pp.
          <fpage>93</fpage>
          -
          <lpage>118</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref66">
        <mixed-citation>
          [66]
          <string-name>
            <given-names>O.</given-names>
            <surname>Koblet</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R. S.</given-names>
            <surname>Purves</surname>
          </string-name>
          ,
          <article-title>From online texts to landscape character assessment: Collecting and analysing first-person landscape perception computationally</article-title>
          ,
          <source>Landsc. Urban Plan</source>
          .
          <volume>197</volume>
          (
          <year>2020</year>
          )
          <article-title>103757</article-title>
          . doi:
          <volume>10</volume>
          . 1016/j.landurbplan.
          <year>2020</year>
          .
          <volume>103757</volume>
          .
        </mixed-citation>
      </ref>
      <ref id="ref67">
        <mixed-citation>
          [67]
          <string-name>
            <surname>M. C. Hartmann</surname>
            ,
            <given-names>O.</given-names>
          </string-name>
          <string-name>
            <surname>Koblet</surname>
            ,
            <given-names>M. F.</given-names>
          </string-name>
          <string-name>
            <surname>Baer</surname>
            ,
            <given-names>R. S.</given-names>
          </string-name>
          <string-name>
            <surname>Purves</surname>
          </string-name>
          , Automated motif identification:
          <article-title>Analysing flickr images to identify popular viewpoints in europe's protected areas</article-title>
          ,
          <source>Journal of Outdoor Recreation and Tourism</source>
          <volume>37</volume>
          (
          <year>2022</year>
          )
          <article-title>100479</article-title>
          . doi:
          <volume>10</volume>
          .1016/ j.jort.
          <year>2021</year>
          .
          <volume>100479</volume>
          .
        </mixed-citation>
      </ref>
      <ref id="ref68">
        <mixed-citation>
          [68]
          <string-name>
            <given-names>G.</given-names>
            <surname>Marzano</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Lizut</surname>
          </string-name>
          ,
          <string-name>
            <given-names>L. O.</given-names>
            <surname>Siguencia</surname>
          </string-name>
          ,
          <article-title>Crowdsourcing solutions for supporting urban mobility</article-title>
          ,
          <source>Procedia Comput. Sci</source>
          .
          <volume>149</volume>
          (
          <year>2019</year>
          )
          <fpage>542</fpage>
          -
          <lpage>547</lpage>
          . doi:
          <volume>10</volume>
          .1016/j. procs.
          <year>2019</year>
          .
          <volume>01</volume>
          .174.
        </mixed-citation>
      </ref>
      <ref id="ref69">
        <mixed-citation>
          [69]
          <string-name>
            <given-names>D.</given-names>
            <surname>Nevo</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Kotlarsky</surname>
          </string-name>
          ,
          <article-title>Crowdsourcing as a strategic IS sourcing phenomenon: Critical review and insights for future research</article-title>
          ,
          <source>J. Strat. Inf. Syst</source>
          .
          <volume>29</volume>
          (
          <year>2020</year>
          )
          <article-title>101593</article-title>
          . doi:
          <volume>10</volume>
          .1016/j.jsis.
          <year>2020</year>
          .
          <volume>101593</volume>
          .
        </mixed-citation>
      </ref>
      <ref id="ref70">
        <mixed-citation>
          [70]
          <string-name>
            <given-names>P.</given-names>
            <surname>Abrahamsson</surname>
          </string-name>
          ,
          <string-name>
            <given-names>N.</given-names>
            <surname>Oza</surname>
          </string-name>
          , M. T. Siponen,
          <article-title>Agile software development methods: A comparative Review1</article-title>
          ,
          <source>in: Agile Software Development</source>
          , Springer Berlin Heidelberg, Berlin, Heidelberg,
          <year>2010</year>
          , pp.
          <fpage>31</fpage>
          -
          <lpage>59</lpage>
          . doi:
          <volume>10</volume>
          .1007/978-3-
          <fpage>642</fpage>
          -12575-1\_3.
        </mixed-citation>
      </ref>
      <ref id="ref71">
        <mixed-citation>
          [71]
          <string-name>
            <given-names>L.</given-names>
            <surname>Williams</surname>
          </string-name>
          ,
          <string-name>
            <given-names>G.</given-names>
            <surname>Brown</surname>
          </string-name>
          , A. Meltzer,
          <string-name>
            <given-names>N.</given-names>
            <surname>Nagappan</surname>
          </string-name>
          , Scrum + engineering practices:
          <article-title>Experiences of three microsoft teams</article-title>
          ,
          <source>International Symposium on Empirical Software Engineering and Measurement</source>
          (
          <year>2011</year>
          )
          <fpage>463</fpage>
          -
          <lpage>471</lpage>
          . doi:
          <volume>10</volume>
          .1109/esem.
          <year>2011</year>
          .
          <volume>65</volume>
          .
        </mixed-citation>
      </ref>
      <ref id="ref72">
        <mixed-citation>
          [72]
          <string-name>
            <given-names>D.</given-names>
            <surname>Jones</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Nassehi</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C.</given-names>
            <surname>Snider</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Gopsill</surname>
          </string-name>
          ,
          <string-name>
            <given-names>P.</given-names>
            <surname>Rosso</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R.</given-names>
            <surname>Real</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Goudswaard</surname>
          </string-name>
          ,
          <string-name>
            <given-names>B.</given-names>
            <surname>Hicks</surname>
          </string-name>
          ,
          <article-title>Towards integrated version control of virtual and physical artefacts in new product development: Inspirations from software engineering and the digital twin paradigm</article-title>
          ,
          <source>Procedia CIRP 100</source>
          (
          <year>2021</year>
          )
          <fpage>283</fpage>
          -
          <lpage>288</lpage>
          . doi:
          <volume>10</volume>
          .1016/j.procir.
          <year>2021</year>
          .
          <volume>05</volume>
          .121.
        </mixed-citation>
      </ref>
      <ref id="ref73">
        <mixed-citation>
          [73]
          <string-name>
            <given-names>H. K.</given-names>
            <surname>Flora</surname>
          </string-name>
          ,
          <string-name>
            <given-names>X.</given-names>
            <surname>Wang</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S. V.</given-names>
            <surname>Chande</surname>
          </string-name>
          ,
          <article-title>An investigation into mobile application development processes: Challenges and best practices</article-title>
          ,
          <source>Int. J. Mod. Educ. Comput. Sci. 6</source>
          (
          <issue>2014</issue>
          )
          <fpage>1</fpage>
          -
          <lpage>9</lpage>
          . doi:
          <volume>10</volume>
          .5815/ijmecs.
          <year>2014</year>
          .
          <volume>06</volume>
          .01.
        </mixed-citation>
      </ref>
      <ref id="ref74">
        <mixed-citation>
          [74]
          <string-name>
            <given-names>Y.</given-names>
            <surname>Wang</surname>
          </string-name>
          ,
          <string-name>
            <given-names>Y.</given-names>
            <surname>Huang</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Li</surname>
          </string-name>
          ,
          <string-name>
            <surname>J. Zhang,</surname>
          </string-name>
          <article-title>The efect of mobile applications' initial loading pages on users' mental state and behavior</article-title>
          ,
          <source>Displays</source>
          <volume>68</volume>
          (
          <year>2021</year>
          )
          <article-title>102007</article-title>
          . doi:
          <volume>10</volume>
          .1016/j.displa.
          <year>2021</year>
          .
          <volume>102007</volume>
          .
        </mixed-citation>
      </ref>
      <ref id="ref75">
        <mixed-citation>
          [75]
          <string-name>
            <given-names>M. E.</given-names>
            <surname>Rana</surname>
          </string-name>
          , E. Khonica,
          <article-title>Impact of design principles and patterns on software flexibility: An experimental evaluation using lfexible point (FXP)</article-title>
          ,
          <source>J. Comput. Sci</source>
          .
          <volume>17</volume>
          (
          <year>2021</year>
          )
          <fpage>624</fpage>
          -
          <lpage>638</lpage>
          . doi:
          <volume>10</volume>
          .3844/jcssp.
          <year>2021</year>
          .
          <volume>624</volume>
          .638.
        </mixed-citation>
      </ref>
      <ref id="ref76">
        <mixed-citation>
          [76]
          <string-name>
            <given-names>X.</given-names>
            <surname>Zhong</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Xu</surname>
          </string-name>
          ,
          <article-title>Measuring the efect of game updates on player engagement: A cue from DOTA2, Entertain</article-title>
          . Comput.
          <volume>43</volume>
          (
          <year>2022</year>
          )
          <article-title>100506</article-title>
          . doi:
          <volume>10</volume>
          .1016/j.entcom.
          <year>2022</year>
          .
          <volume>100506</volume>
          .
        </mixed-citation>
      </ref>
      <ref id="ref77">
        <mixed-citation>
          [77]
          <string-name>
            <given-names>C.</given-names>
            <surname>Guo</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Wei</surname>
          </string-name>
          ,
          <string-name>
            <given-names>Y.</given-names>
            <surname>Chen</surname>
          </string-name>
          ,
          <source>Cognitive processing of information visualization</source>
          ,
          <source>Encyclopedia of Computer Graphics and Games</source>
          (
          <year>2018</year>
          )
          <fpage>1</fpage>
          -
          <lpage>12</lpage>
          . doi:
          <volume>10</volume>
          .1007/ 978-3-
          <fpage>319</fpage>
          -08234-9\_
          <fpage>95</fpage>
          -
          <lpage>1</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref78">
        <mixed-citation>
          [78]
          <string-name>
            <given-names>C.</given-names>
            <surname>Certomà</surname>
          </string-name>
          ,
          <string-name>
            <given-names>F.</given-names>
            <surname>Corsini</surname>
          </string-name>
          ,
          <string-name>
            <given-names>F.</given-names>
            <surname>Rizzi</surname>
          </string-name>
          ,
          <article-title>Crowdsourcing urban sustainability. data, people and technologies in participatory governance</article-title>
          ,
          <source>Futures</source>
          <volume>74</volume>
          (
          <year>2015</year>
          )
          <fpage>93</fpage>
          -
          <lpage>106</lpage>
          . doi:
          <volume>10</volume>
          . 1016/j.futures.
          <year>2014</year>
          .
          <volume>11</volume>
          .006.
        </mixed-citation>
      </ref>
      <ref id="ref79">
        <mixed-citation>
          [79]
          <string-name>
            <given-names>A.</given-names>
            <surname>Taeihagh</surname>
          </string-name>
          ,
          <article-title>Crowdsourcing: a new tool for policy-making?</article-title>
          ,
          <source>Policy Sci</source>
          .
          <volume>50</volume>
          (
          <year>2017</year>
          )
          <fpage>629</fpage>
          -
          <lpage>647</lpage>
          . doi:
          <volume>10</volume>
          .1007/ s11077-017-9303-3.
        </mixed-citation>
      </ref>
      <ref id="ref80">
        <mixed-citation>
          [80]
          <string-name>
            <given-names>T.</given-names>
            <surname>Aitamurto</surname>
          </string-name>
          ,
          <string-name>
            <given-names>K.</given-names>
            <surname>Chen</surname>
          </string-name>
          ,
          <article-title>The value of crowdsourcing in public policymaking: epistemic, democratic and economic value</article-title>
          ,
          <source>Theory Pract. Legis</source>
          .
          <volume>5</volume>
          (
          <year>2017</year>
          )
          <fpage>55</fpage>
          -
          <lpage>72</lpage>
          . doi:
          <volume>10</volume>
          .1080/20508840.
          <year>2017</year>
          .
          <volume>1282665</volume>
          .
        </mixed-citation>
      </ref>
      <ref id="ref81">
        <mixed-citation>
          [81]
          <string-name>
            <given-names>S.</given-names>
            <surname>Barns</surname>
          </string-name>
          ,
          <article-title>Smart cities and urban data platforms: Designing interfaces for smart governance</article-title>
          ,
          <source>City Cult. Soc</source>
          .
          <volume>12</volume>
          (
          <year>2018</year>
          )
          <fpage>5</fpage>
          -
          <lpage>12</lpage>
          . doi:
          <volume>10</volume>
          .1016/j.ccs.
          <year>2017</year>
          .
          <volume>09</volume>
          .006.
        </mixed-citation>
      </ref>
      <ref id="ref82">
        <mixed-citation>
          [82]
          <string-name>
            <given-names>H.</given-names>
            <surname>Niu</surname>
          </string-name>
          ,
          <string-name>
            <given-names>E. A.</given-names>
            <surname>Silva</surname>
          </string-name>
          ,
          <article-title>Crowdsourced data mining for urban activity: Review of data sources, applications, and methods</article-title>
          ,
          <source>J. Urban Plan. Dev</source>
          .
          <volume>146</volume>
          (
          <year>2020</year>
          )
          <article-title>04020007</article-title>
          . doi:
          <volume>10</volume>
          .1061/(asce)up.
          <fpage>1943</fpage>
          -
          <volume>5444</volume>
          .
          <fpage>0000566</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref83">
        <mixed-citation>
          [83]
          <string-name>
            <given-names>S.</given-names>
            <surname>Schaal</surname>
          </string-name>
          ,
          <article-title>Location-based games for geography and environmental education, in: Geography Education in the Digital World, 1st edition ed</article-title>
          .,
          <source>Routledge</source>
          ,
          <year>2020</year>
          , pp.
          <fpage>168</fpage>
          -
          <lpage>178</lpage>
          . doi:
          <volume>10</volume>
          .4324/
          <fpage>9780429274909</fpage>
          -
          <lpage>15</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref84">
        <mixed-citation>
          [84]
          <string-name>
            <given-names>J.</given-names>
            <surname>Sheahan</surname>
          </string-name>
          ,
          <string-name>
            <given-names>H.</given-names>
            <surname>Davies</surname>
          </string-name>
          ,
          <string-name>
            <surname>L. Hjorth,</surname>
          </string-name>
          <article-title>The art of tacit learning in serious location-based</article-title>
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>