<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta>
      <journal-title-group>
        <journal-title>Journal of Business Ethics
155.9 (2019). https://doi.org/10.1007/s10551</journal-title>
      </journal-title-group>
    </journal-meta>
    <article-meta>
      <title-group>
        <article-title>Gamifying responsibly: Sustainable HCI for the future</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Georgina Guillén Mandujano</string-name>
          <email>georgina.guillen@tuni.fi</email>
          <xref ref-type="aff" rid="aff0">0</xref>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Nevena Sicevic</string-name>
          <email>nevena.sicevic@tuni.fi</email>
          <xref ref-type="aff" rid="aff0">0</xref>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Juho Hamari</string-name>
          <email>juho.hamari@tuni.fi</email>
          <xref ref-type="aff" rid="aff0">0</xref>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>8th International GamiFIN Conference 2024</institution>
          ,
          <addr-line>GamiFIN 2024</addr-line>
        </aff>
        <aff id="aff1">
          <label>1</label>
          <institution>Gamification Group. Tampere University</institution>
          ,
          <addr-line>Kalevantie 4, 33100 Tampere</addr-line>
          ,
          <country country="FI">Finland</country>
        </aff>
      </contrib-group>
      <pub-date>
        <year>2021</year>
      </pub-date>
      <volume>3</volume>
      <fpage>491</fpage>
      <lpage>506</lpage>
      <abstract>
        <p>With the ever-rising consideration for holistic sustainability in contemporary technology, there is a dearth of knowledge of how pursuits among design can and have been responsibly re-configured towards these ends. Therefore, in this paper, we explore the potential of responsibly implementing gamification as an innovation in sustainable consumption apps (SCAs) to deepen the understanding and knowledge about the development process of sustainable human-computer interaction (SHCI). We utilize gamification design practice as our contextual case as it both imbues sustainability in the design ethos and as the modus operandi of the technology itself while simultaneously being susceptible to several moral hazards. This qualitative study employs the prism of responsible research and innovation (RRI) dimensions as the kaleidoscopic lens for analyzing the data gathered through key-informant, semi-structured interviews among 21 SCA creators. Moreover, the study links the SCA creators' perceptions of risks for the users with a series of app users' needs to highlight areas of concern to gamify SCAs responsibly. The overview of all findings is presented as recommendations for HCI practitioners and interested stakeholders to use RRI dimensions as a guideline to make informed decisions for the responsible development of SCA. These recommendations include considering sustainability values and ethics as a prerequisite for decision-making from the conceptualization phase onwards, implementing multi-stakeholder, participatory design processes, cross-cultural cooperation to enable socially desirable outcomes, and developing and implementing responsive accountability practices to nurture a sense of shared responsibility.</p>
      </abstract>
      <kwd-group>
        <kwd>eol&gt;Sustainable human-computer interaction</kwd>
        <kwd>responsible research and innovation</kwd>
        <kwd>gamification</kwd>
        <kwd>sustainable consumption</kwd>
        <kwd>mobile apps 1</kwd>
      </kwd-group>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>1. Introduction</title>
      <p>
        We are contemporarily living in a global culture where
sustainability is emphasized as the “development that
meets the needs of the present without compromising
the ability of future generations to meet their own
needs” ([
        <xref ref-type="bibr" rid="ref1">1</xref>
        ] n.p.), applied to all spheres of human
activities, calling for collaborations between social and
natural sciences to understand their interactions [
        <xref ref-type="bibr" rid="ref2">2</xref>
        ].
Part of these processes rely on technological solutions
to make everyday choices, from increasing an efficient
use of natural resources to facilitating the
advancement of interconnected, digital societies in
ways that reduce inequality and wellbeing gaps, both
for people and the environment.
      </p>
      <p>The people behind these technologies have a
twofold challenge: develop solutions that enable users
to act sustainably; and, behave themselves more
responsibly when creating these solutions. Hence,
Sustainable Human-Computer Interaction (SHCI) has
been gaining relevance for over a decade as a research
0000-0002-2462-0082 (G. Guillén Mandujano);
0000-0002-05798977 (N. Sicevic); 0000-0002-6571-588X (J. Hamari)
© 2024 Copyright for this paper by its authors. The use permitted under
Creative Commons License Attribution 4.0 International (CC BY 4.0).</p>
      <p>
        CEUR Workshop Proceedings (CEUR-WS.org)
discipline through two categories: sustainability in
design and sustainability through design [[
        <xref ref-type="bibr" rid="ref3">3</xref>
        ], [
        <xref ref-type="bibr" rid="ref4">4</xref>
        ], [
        <xref ref-type="bibr" rid="ref5">5</xref>
        ],
[
        <xref ref-type="bibr" rid="ref6">6</xref>
        ]]. The former concerns itself with efficiency and
digital services, like enabling circularity through the
products’ physical attributes. The latter relates to
applying HCI to support decision-making processes to
lead more sustainable lifestyles. Due to the rapidly
evolving nature of HCI, it is necessary to find
frameworks that help operationalizing sustainability
[
        <xref ref-type="bibr" rid="ref7">7</xref>
        ] and help evaluating the usability aspects of the
approaches related to it; thus, SCHI calls for a critical,
yet all-encompassing approach to innovation,
particularly in the technological front [
        <xref ref-type="bibr" rid="ref8">8</xref>
        ].
      </p>
      <p>
        There are several HCI/technological developments
where the solution creators try to act more
responsibly and orient their efforts towards tackling
sustainability grand challenges holistically [
        <xref ref-type="bibr" rid="ref6">6</xref>
        ],
identifying appropriate appraisal methods for SHCI
today and in the future, including ways to
communicate sustainability solutions [
        <xref ref-type="bibr" rid="ref5">5</xref>
        ]. Among
these approaches, there is an increased reliance on the
application of persuasive systems, which in most
instances are called eco-feedback systems [[
        <xref ref-type="bibr" rid="ref5">5</xref>
        ], [
        <xref ref-type="bibr" rid="ref7">7</xref>
        ], [
        <xref ref-type="bibr" rid="ref9">9</xref>
        ]]
or “technology that provides feedback on individual or
group behaviors with a goal of reducing environmental
impact” ([
        <xref ref-type="bibr" rid="ref10">10</xref>
        ], p. 1999). Given their characteristics,
these eco-feedback systems can be considered an
environmentally focused representation of
gamification – an intentional process designed to
afford positive experiences, skills, and practices
similar to those of games through any activity, system,
service, product, or organizational structure [
        <xref ref-type="bibr" rid="ref11">11</xref>
        ].
However, due to their potential to affect human
behavior, social and mental wellbeing, gamification
and other similar approaches are often questioned as
a responsible practice [[
        <xref ref-type="bibr" rid="ref12">12</xref>
        ], [
        <xref ref-type="bibr" rid="ref13">13</xref>
        ]]. SCHI research
shows that eco-feedback as a persuasive system seems
to reduce the understanding of sustainability as a
matter of negotiations rather than informed-based
change [
        <xref ref-type="bibr" rid="ref7">7</xref>
        ], and the consideration of long-term or
negative impacts are largely overlooked. Therefore, in
the context of SHCI, the design and implementation
approaches of gamification as innovation should
consider potential harmful impacts and call for
inclusion of responsible practices to address the
challenges inherent to bringing together different
stakeholder interests.
      </p>
      <p>To this end, the present study concerns itself with
the value-based considerations for creating gamified
sustainable consumption apps according to the
dimensions of responsible research and innovation
(RRI) endorsed by the European Union. Beckoning the
sustainable consumption app (SCA) creators to
consider their apps' intended and unintended impacts,
we hypothesize that including gamification as an
innovation under RRI dimensions into the app design
process can contribute to a value-based design of SCA
and more sustainable human-computer interaction
practices. Thus, our research question is: How can SCA
creators responsibly integrate value-driven
gamification as an innovation into their design and
implementation processes?</p>
    </sec>
    <sec id="sec-2">
      <title>2. Theoretical background</title>
      <p>
        The current HSCI literature, while calling for
multidisciplinary, intersectoral participatory
processes, tends to focus on the user perspective, the
functionality of the designs, and their expected
impacts [[
        <xref ref-type="bibr" rid="ref4">4</xref>
        ], [
        <xref ref-type="bibr" rid="ref5">5</xref>
        ], [
        <xref ref-type="bibr" rid="ref6">6</xref>
        ], [
        <xref ref-type="bibr" rid="ref8">8</xref>
        ], [
        <xref ref-type="bibr" rid="ref9">9</xref>
        ], [
        <xref ref-type="bibr" rid="ref14">14</xref>
        ], [
        <xref ref-type="bibr" rid="ref15">15</xref>
        ]]; hardly
addressing the mindsets of the creators or
acknowledging their individual motivations to become
sustainability activists through HCI solutions.
Therefore, in this study we investigated how SHCI
practitioners, represented by the creators of
sustainable consumption apps deal, and understand
their responsibility to implement gamification as an
innovation, addressing the call for practical
approaches to translate responsible research and
innovation from a conceptual aspiration to an
implementable strategy, pointing at the need to share
the reasons and methodological decisions from the
early design stages [[
        <xref ref-type="bibr" rid="ref16">16</xref>
        ], [
        <xref ref-type="bibr" rid="ref17">17</xref>
        ]]. Hence, this section
elaborates on the theoretical frameworks used for our
research.
      </p>
      <sec id="sec-2-1">
        <title>2.1. Gamification as an innovation</title>
        <p>
          The eco-feedback activities described in existing SHCI
research usually present a skewed understanding of
sustainability; this is a focus on environmental impacts
and it is usually applied to energy-related activities
[[
          <xref ref-type="bibr" rid="ref4">4</xref>
          ], [
          <xref ref-type="bibr" rid="ref7">7</xref>
          ]]. However, the literature shows a myriad of
formalized principles for the application of persuasive
approaches as part of the design [
          <xref ref-type="bibr" rid="ref4">4</xref>
          ]. Considering
ecofeedback systems as an expression of gamification
opens a multidimensional and synergistic approach to
innovation processes for SHCI since gamification can
be applied through all its phases (search/ideation,
selection, implementation and capture) [
          <xref ref-type="bibr" rid="ref18">18</xref>
          ].
        </p>
        <p>
          Specifically, the existing literature on the subject
portrays gamification as an innovation through three
processes: i) investigation – gamification that reveals
problems; ii) induction – gamification to stimulate
novel behavior; and, iii) intervention – gamification
that transforms processes for improved effectiveness
and engagement [
          <xref ref-type="bibr" rid="ref19">19</xref>
          ].
        </p>
        <p>
          The first type, investigation, is mainly based in
cocreative processes, such as crowdsourcing, to capture
information and transform it into a desired goal, for
example improved health systems, or the development
of more resource-efficient technologies [
          <xref ref-type="bibr" rid="ref19">19</xref>
          ].
Gamification as an induction process is meant to
stimulate innovative behaviors, usually introducing
fantasy elements and challenges of collaborative
behavior, and it is applied to animate scientific
teaching or embedding critical thinking into curricula,
for example [
          <xref ref-type="bibr" rid="ref19">19</xref>
          ]. With the third type, intervention,
gamification acts as a facilitator to enhance an existing
function and it is largely applied in education and
organizational change activities to motivate
participants to engage in actions of change [
          <xref ref-type="bibr" rid="ref19">19</xref>
          ].
        </p>
        <p>
          Regardless of their type, the gamification as
innovation implementation processes often convey
many layers, from correctly identifying the problem to
gamify (e.g. climate change), to the capabilities of
implementing a gamified strategy (e.g. having enough
knowledge of gamification to design a long-term
strategy). These nuances may curtail the innovation
processes, posing a challenge to understanding
gamification as innovation perspective, its integration
potential, and overall applicability throughout the
innovation process [[
          <xref ref-type="bibr" rid="ref18">18</xref>
          ], [
          <xref ref-type="bibr" rid="ref19">19</xref>
          ]]. Moreover, whether
presented as eco-feedback, or as gamification, there is
a constant concern about the extent that applying
persuasive technologies can lead to unsustainable
behaviors [
          <xref ref-type="bibr" rid="ref7">7</xref>
          ], a risk that SHCI practitioners should
always bear in mind. Therefore, recognizing
gamification as an innovation demands examining
gamification design frameworks that enable
responsible, value-driven and practice-based
considerations. To make this possible, [
          <xref ref-type="bibr" rid="ref20">20</xref>
          ] encourage
the creators of gamified approaches to only use
systems they would be willing to be persuaded with
themselves, and action that attains SCHI practitioners
to acknowledge their responsibility towards the users
whose lifestyles they intend to change [
          <xref ref-type="bibr" rid="ref5">5</xref>
          ].
        </p>
      </sec>
      <sec id="sec-2-2">
        <title>2.2. Responsible Research and</title>
      </sec>
      <sec id="sec-2-3">
        <title>Innovation (RRI)</title>
        <p>
          RRI elaborates on the notion of trust building between
individuals and technological innovations, as it places
societal needs at the core of research and innovation,
highlighting the responsibility factors and their
alignment with society's values. RRI intends to secure
societal acceptance of new technologies and trust in
science [
          <xref ref-type="bibr" rid="ref21">21</xref>
          ] through multi-stakeholder collaboration
under principles of transparency and mutual
responsiveness [
          <xref ref-type="bibr" rid="ref22">22</xref>
          ]. Moreover, RRI seeks to present
innovation beyond mere technological development
[
          <xref ref-type="bibr" rid="ref23">23</xref>
          ], acknowledging the existence of uncertainties and
unintended consequences, which all involved
stakeholders should be aware of and prepared to
respond to. Although the European Union has been
endorsing programs and formulating policies
addressing technology-knowledge concerns for over a
decade, the implementation of RRI in other spheres,
such as non-for-profits, business, and corporate
responsibility practices, is still in a very early stage
[[
          <xref ref-type="bibr" rid="ref24">24</xref>
          ], [
          <xref ref-type="bibr" rid="ref25">25</xref>
          ]]. This situation calls for value-conscious
frameworks to facilitate the integration of RRI
dimensions into the design and development practices
of technological innovations such as mobile apps.
        </p>
        <p>
          In their seminal work about ethical implications for
RRI in the information and technologies field, [
          <xref ref-type="bibr" rid="ref26">26</xref>
          ]
categorize RRI as a “meta-responsibility” that aligns
the existing network of responsibilities in the
Information and Communication Technologies (ICT)
sector “defining socially desirable consequences that
existing responsibilities can work toward and develop
responsibility relationships that ensure that the
achievement of such desired aims is possible” ([
          <xref ref-type="bibr" rid="ref26">26</xref>
          ], p.
202). RRI frameworks are built on the dimensions of
anticipation, reflexivity, inclusion, responsiveness, and
care [[
          <xref ref-type="bibr" rid="ref23">23</xref>
          ], [
          <xref ref-type="bibr" rid="ref27">27</xref>
          ]], elements that this study proposes to
integrate into the design processes of sustainable
consumption apps, thus contributing to the field of
sustainable human computer interaction.
        </p>
        <p>
          [
          <xref ref-type="bibr" rid="ref26">26</xref>
          ] propose the “4P” approach to identify and
engage with the ethical implications of ICT. These Ps
are product, process, purpose, and people, and to fully
integrate RRI into each of these, it is crucial to facilitate
processes and indicators to monitor awareness,
implementation, and assessment, based on RRI values
and norms to facilitate their integration along the
value chain [
          <xref ref-type="bibr" rid="ref28">28</xref>
          ]. On the basis thereof, value-conscious
approaches to gamification design and
implementation are paramount for enabling the
contexts where RRI actions can take place, like in the
case of mobile apps designed to help their users live
more sustainably.
        </p>
      </sec>
      <sec id="sec-2-4">
        <title>2.3. The Sustainable Gamification</title>
      </sec>
      <sec id="sec-2-5">
        <title>Design (SGD) framework and app users’ needs</title>
        <p>
          The integration of gamification into the design process,
is a value-driven endeavor as these values take the
shape of engagement, learning, and collaboration,
shaping behaviors, empowerment, transformation,
provision of analytics, and fun [
          <xref ref-type="bibr" rid="ref29">29</xref>
          ]. As such, every
value is also prone to be affected by situations or issues
that destroy them, for example, when using
gamification as a persuasion tool, which conveys
several moral and ethical challenges. To address these
challenges, [
          <xref ref-type="bibr" rid="ref29">29</xref>
          ] proposed the Sustainable
Gamification Design (SGD) framework, a conceptual
model that introduces a frame of values and ethics to
manage the potential negative impacts of
gamification’s value destroyers. Although originally
developed to support the design of gamification
strategies for organizations, the SGD is a human-based
approach to gamification design that reflects the call
for the ethical and responsible design of mobile apps.
        </p>
        <p>
          [
          <xref ref-type="bibr" rid="ref30">30</xref>
          ] bring a more detailed account of the users’
needs and wishes for sustainably developing apps,
bearing in mind some of the risks that concern users
the most. The requirements for a sustainable design of
apps proposed by [
          <xref ref-type="bibr" rid="ref30">30</xref>
          ] include attractiveness,
efficiency, accuracy, and value for money, and strongly
emphasize security and privacy, both as a safety
concern and a risk area for using any app. These
concerns are reflected in each of the gamification value
destroyers distinguished by [
          <xref ref-type="bibr" rid="ref29">29</xref>
          ], which are noted as
gamification leading to coercion, data leaking,
channeling, norming, disempowering, misrepresenting,
and providing inauthentic and shallow accounts. These
value destroyers also stand against everything RRI
represents, posing a challenge to its implementation.
For example, in the case of gamified mobile apps
enabling sustainable consumption practices (SCA)
there is no evidence of a long-term behavioral change;
however, there is an overriding focus on
environmental issues and plenty of gamification
practices with unclear purposes [
          <xref ref-type="bibr" rid="ref31">31</xref>
          ]. Nonetheless, the
benefits of applying gamification to motivate more
responsible consumption practices are exemplified by
improved wellbeing conditions facilitated by mobile
apps purposefully designed to this end [[
          <xref ref-type="bibr" rid="ref14">14</xref>
          ], [
          <xref ref-type="bibr" rid="ref15">15</xref>
          ]].
However, they come with many cautionary tales about
unintended impacts and behaviors, which attain
directly to the need of implementing practices of
responsibility and trust-building in the development of
these apps.
        </p>
      </sec>
    </sec>
    <sec id="sec-3">
      <title>3. Methods</title>
      <sec id="sec-3-1">
        <title>3.1. Approach</title>
        <p>
          When assessed against the framework of the United
Nations Sustainable Development Goals (SDGs), the
global agenda for acting today toward a sustainable
future [
          <xref ref-type="bibr" rid="ref32">32</xref>
          ], most of the research about HCI and
sustainability happens in the field of Responsible
Consumption and Production (SDG 12), mainly
addressing the management and efficient use of
natural resources, and reducing waste generation [
          <xref ref-type="bibr" rid="ref7">7</xref>
          ];
thus, promoting more environmentally-friendly and,
potentially, more sustainable consumption choices.
Moreover, mobile applications are the most used yet
emergent HCI interface to promote sustainable
consumption at the individual level [[
          <xref ref-type="bibr" rid="ref33">33</xref>
          ], [
          <xref ref-type="bibr" rid="ref34">34</xref>
          ]] as they
are known for functions such as marketing,
entertainment, information provision, socialization,
and even intellectual stimulation [[
          <xref ref-type="bibr" rid="ref35">35</xref>
          ], [
          <xref ref-type="bibr" rid="ref36">36</xref>
          ]].
        </p>
        <p>
          Based on the definition of sustainable consumption
behaviors as "individual acts of satisfying needs in
different areas of life by acquiring, using and disposing
goods and services that do not compromise the
ecological and socio-economic conditions of all people
(currently living or in the future) to satisfy their own
needs" ([
          <xref ref-type="bibr" rid="ref2">2</xref>
          ], p. 5), this study considers sustainable
consumption apps (SCA) as the mobile applications
created to enable individual choices that satisfy needs
through different consumption stages without
compromising the living conditions of people and other
species today and in the future. Despite their future
orientation, SCAs often present some gamified
features, ranging from badge collection and
leaderboards to in-app and external rewards, and they
tend to disappear five years or less after their launch
to the market [
          <xref ref-type="bibr" rid="ref31">31</xref>
          ]. The changing landscape of apps for
sustainability leads to questioning the values and
motivations behind the existence of such apps and the
challenges their creators face to make them meet their
objectives. Some of these challenges include issues
such as how researchers and practitioners choose
their sustainable consumption narratives and
gamification elements, leaving open the opportunity to
refine their design strategies and deliver more
comprehensive, systemic (cause-effect)
understandings of sustainable consumption via
gamification.
        </p>
        <p>
          This study considers sustainable consumption
apps (SCA) as mobile applications created with the
intention of enabling behaviors resulting from the
awareness of the impact of today’s consumption
activities and the future wellbeing of societies and the
environment; in other words, what [
          <xref ref-type="bibr" rid="ref2">2</xref>
          ] define as
sustainable consumption behaviors. Understanding
RRI “as an attempt to give a procedural answer to the
question of how to deal with the uncertainties around
innovation” ([
          <xref ref-type="bibr" rid="ref26">26</xref>
          ], p. 203), to answer its research
question, the research is framed according to [
          <xref ref-type="bibr" rid="ref26">26</xref>
          ]’s
“4P’s” of RRI, a framework created to highlight the
purpose and the people behind the innovation and not
only the product and the process, (Table 1) to identify
and engage with the ethical implications of ICT in the
context of gamification as an innovation.
        </p>
        <p>
          SCA creators presenting their accounts of
identified societal consequences beyond the
use of their app (interview results)
Input from app users’ perceived risks and
requirements as presented by [
          <xref ref-type="bibr" rid="ref30">30</xref>
          ].
        </p>
      </sec>
      <sec id="sec-3-2">
        <title>3.2. Data collection and analysis</title>
        <p>
          To initiate the process of identifying the SCA creators’
accounts of gamification-led value destruction and
comparing their concerns with the app users’ requests
and risk perceptions, the first step was to select the
pool of apps whose creators’ opinions will be
examined. The app database from an earlier study [
          <xref ref-type="bibr" rid="ref31">31</xref>
          ]
allowed the selection of the product: 52 apps
representing different stages of popularity and
gamification features. The sample consisted of apps
marketed in Google Play and App Store under the
labels of sustainable consumption, sustainability,
sustainable living, sustainable lifestyles, green
lifestyles and eco-friendly living. Other keywords
related to lifestyles, such as “mindfulness” or
“wellbeing” were not used as they might not relate
directly to consumption practices.
        </p>
        <p>To enable diversity in perspectives, and fulfill the
purpose of this study – contribute to the
implementation of value-oriented, innovative design
practices to advance SHCI – a pool of apps was created
with the following categories, with 13 apps selected for
each: i) highly downloaded and rated (most popular)
gamified apps; ii) apps that disappeared through the
2021 analysis; iii) apps that had more gamification
elements than the average of apps analyzed in 2021;
iv) new apps that appeared in the market after the
2021 review, and apps that were not analyzed in 2021
because they were not gamified. The last group
(nongamified apps) was included to learn about the
considerations for not implementing gamification. The
21 interviewees represent 2 apps from group 1; 6 of
group 2; 6 of group 3; and 7 of group 4, three of which
are not gamified.</p>
        <p>
          The participants are considered key informants
due to their close relationship to the research subject
[
          <xref ref-type="bibr" rid="ref37">37</xref>
          ]. The process to contact the interviewees consisted
of a) reviewing the app information available on the
online platform, the app’s website (if applicable) or the
app itself. These channels contain the name of an
organization or a person behind the app. 24 of the 27
people who replied to the invitation to join the study
were contacted via their Linked-in profiles or personal
email address (two of them have it as part of the apps’
information), two replied to the “info”-general email
address, and only one contacted us after we submitted
a meeting request through their online contact form.
All participants were provided with a description of
the research project, the data management guidelines,
and an informed consent form. In the end, 21 app
creators – the people behind the conceptualization,
implementation, and maintenance of the app, who
could also be the technical developers but not
necessarily – were interviewed via Zoom and MS
Teams during April and August 2022. Regardless of
their role in the organization behind the app, the 21
creators interviewed are decision-makers for the app’s
survival, meaning they are directly responsible for its
maintenance and online presence. The questionnaire
for the semi-structured interviews was reviewed and
validated by four topic experts, from China, Finland,
Mexico, and the Netherlands, respectively. The
interview guideline comprised 4 parts. The first
focused on the creators’ background, their
understanding of sustainability and the reason for
creating an app to act upon this understanding. The
second part zoomed into the expected sustainability
impacts of the apps and the role gamification played
within, including elaborating on the notions of ethics
and responsibility as a creator of a
sustainabilityoriented solution. The questions of the third section
provided insights into the tensions and dilemmas of
managing a mobile app in a highly competitive market;
while the fourth section offered the opportunity to
reflect on their overall learning journeys. The full
interview guideline is available in [
          <xref ref-type="bibr" rid="ref38">38</xref>
          ].
        </p>
        <p>
          After the interviews, the answers were
anonymized and coded which allowed us to identify
the areas of concern according to the SGD value
destroyers [
          <xref ref-type="bibr" rid="ref29">29</xref>
          ]. The diverse understandings of
responsibility and risks presented by the creators,
compared with the user requests from [
          <xref ref-type="bibr" rid="ref30">30</xref>
          ]], also shed
light on the most and least explored RRI dimensions,
helping to draft what now can be used as a design
guideline for SCA creators. Figure 1 summarizes the
process followed for this study.
        </p>
      </sec>
    </sec>
    <sec id="sec-4">
      <title>4. Results</title>
      <p>This study is framed according to the 4P approach for
responsibly identifying and engaging with the ethical
implications of sustainability through design via
gamification as a threefold innovation; it analyzes the
ethical considerations behind mobile apps created to
encourage sustainable consumption practices and
contribute to the growth of innovative design practices
in sustainable human-computer interaction and
sustainable human-computer interaction as a research
field. The resulting analysis intended to emphasize the
SCA creators’ responsibility when choosing to
implement gamification in their apps. This section
showcases the findings of the interviews with 21 SCA
creators, highlighting both their main concerns
regarding implementing gamification and approaches
to act according to the RRI dimensions to tackle these
value destroyers.</p>
      <sec id="sec-4-1">
        <title>4.1. Identification of SCA creators’ concerns of gamification as innovation through the SGD framework</title>
        <p>
          The question about approaches to persuade people
to use their apps and meet their ultimate objectives
helped the interviewees elaborate on why applying
gamification and their choices to it related. Figure 2
provides an overview of the answers. Analyzing their
accounts through the value destroyers of the SGD
showed that the main concern about using
gamification in SCA is related to the potential human
agency loss, a value destroyer related to how
humancomputer interactions attribute agency to the
computer rather than to the person, hence reducing
the individual’s enjoyment and autonomy, depriving
the user of their freedom to make decisions. This
finding is consistent with what [
          <xref ref-type="bibr" rid="ref39">39</xref>
          ] note as inhibitors
of rational self-reflection that undermine the users’
autonomy that lead to considering gamification as a
manipulative strategy. Considering that 16 of the 21
creators indicated their apps intend to support
individual choice-making, it is not surprising that their
primary concern is designing apps that make people
realize they have the power to choose. Nonetheless,
the issue of why the app facilitated individual agency
presented polarized arguments. Five creators declared
their apps intend to provide information in a way that
people do not feel judged or preached to, that users
were willing to act and only need some guidance for
their efforts; “People don’t want to feel they are being
told what to do or being talked down to or lectured”
(creator 1). On the other hand, five creators declared
that the apps were needed because users would not do
anything on their own even if having information, so
the app gave them an easy way to be active; “Most
people are lazy, they need to be told what to do so they
realize changing is not that difficult” (creator 2).
        </p>
        <p>
          Current studies addressing the ethical aspects of
gamification tend to highlight issues of data
management and privacy as factors that may inhibit
the uptake of gamified SHCI solutions [[
          <xref ref-type="bibr" rid="ref40">40</xref>
          ], [
          <xref ref-type="bibr" rid="ref41">41</xref>
          ]]
therefore, it is not surprising that leaky containers was
the second issue of high concern. Most creators had
data management and privacy-control mechanisms in
place, although they varied greatly. For example, two
apps do not require signing up or registration of any
sort. In contrast, others allowed registration via social
media accounts, which means that the users abide by
the privacy rules of these providers and not the apps.
Eighteen creators claimed not to collect any personal
data, having only anonymized user statistics; “We only
want to see how often the app is used and where. We
have a feedback form, and our users contact us
constantly; sometimes they sign with their names, but
we don’t store these” (creator 4). Privacy was a
priority focus for the apps targeting underaged users
(2 apps) or involving in-app transactions (6 apps); it is
also part of the value proposition for the apps
partnering with companies: no individual employee
data is collected, the information is all anonymized and
only the company, not even the app’s team, have access
to it; “We provide the technical and content support,
we do not collect personal information of any sort”
(creator 5).
        </p>
        <p>
          Two value destroyers were deemed equally
worrisome (8 creators each); one was the
homogenization of the workforce and the other,
creating an illusion of change. The former is related to
data collection and mining, and it is an area of primary
concern for apps dealing with business partners as
there is a risk that employees will be treated as having
the same level of knowledge or interest in the topic,
obliterating their individual motivations and
experiences in the area of sustainable consumption.
Moreover, this concern reiterates the notion of
separate virtual and real lives, as human actions
become the result of data-driven dictations [42].
However, only one app provides its partners with
anonymized data about the “green consumers” so they
can tailor their sustainability campaigns. While this
may be a foray into the user homogenization territory,
the creator noted that the app’s appeal is to provide
accurate data regarding time, length of actions, and
location, cross-referencing it with non-identifiable
information such as gender and age. The illusion of
change was one of the main arguments for not
applying gamification or doing so very lightly so as not
to convey the idea that using the app was enough to
drive change. This concern reflects what [43] warned
about the risks of using gamification as a hook that can
eventually lead to deception, or even develop addictive
behaviors [[
          <xref ref-type="bibr" rid="ref39">39</xref>
          ], [42], [43]]. Eight creators highlighted
the relevance to clarify to the users that the real impact
happens outside the app and that gamification is just
to assist them in adopting new habits. Another reason
for concern is that gamification may simplify the
magnitude of the problem, so users should be made
aware that the small gratifications provided by the app
are a recognition for their efforts to achieve a more
significant, real-life goal. “A lot of apps out there have
a self-declaration mechanism whereby you can sit on
the sofa and say ‘OK I took a 3min shower and I did
great’ […] with our app we are trying to make actions
verifiable and fairer to all. There are certain
limitations, and this is an area we are working on
because we need to improve it” (creator 6). In some
cases, the creators justified the use of gamification as
the best way to visualize the change; “We decided to
put fun first because the impact comes after people
play the game […] it can happen without the app, true,
but it may not be half as fun” (creator 7).
        </p>
        <p>In a similar vein, the value destroyer of coercive
participation was noted as a risk for people using the
apps as part of an employee engagement strategy, as
they may feel forced to join the program even though
participation must be completely voluntary with no
other incentive from the app than making things more
entertaining and useful. In the workplace context, this
risk also attains to the potential exploitation of
employees [[44], [45]]. Also, two creators noted close
cooperation with their partners to design the app as a
part of their unique value proposition; this is because,
for them, understanding the context and wishes of the
partner helps to prevent implementing features that
could be perceived as coercive; “We have to be very,
very context sensitive. I develop apps for social causes,
you can’t be patronizing, and you can’t just force
people to play and like your app just because you’ve
got the funding for it” (creator 8). Consistent with the
warnings for implementing gamification presented by
[[42], [44]], of four creators who steered clear of
gamification altogether two were highly concerned
about gamification providing a shallow and
unauthentic understanding of the apps and the
problems they try to address. These creators noted
how the “dopamine rush” provided by gamification
would distract the users from the real issue and create
the sensation that things were an easy fix, and even
lead to behaviors like cheating for the sake of getting
the rewards rather than shifting consumption
patterns; “The rewarding system […] may induce
people to start cheating just to keep competing, totally
missing the point of the app” (creator 9). The creators
of the other 2 non-gamified apps noted not knowing
enough about the subject, nor having the resources to
consider gamification as relevant for their apps. Lastly,
the technological whip was the value destroyer SCA
creators are less concerned with, as this is a risk
related to using gamification to maintain
organizational social constructs, which does not apply
to the apps not operating in an organizational
environment. The creators of the seven apps involved
in engagement programs declared their apps are used
for corporate responsibility activities or educational
programs, all of voluntary participation, presenting
concerns similar to those identified by [[44], [45]].</p>
        <p>The analysis of the SCA creators’ concerns about
implementing gamification through the SGD lens
helped to outline some of the strategies and
opportunities for contributing to RRI practices and the
overall SHCI development field.</p>
      </sec>
      <sec id="sec-4-2">
        <title>4.2. Alignment with RRI dimensions</title>
        <p>
          The systematic analysis of gamified SCA of 2022 [
          <xref ref-type="bibr" rid="ref31">31</xref>
          ]
called for the need to integrate socio-ethical issues into
the development of SCA, expanding collaborative
sMtaakrekehtoaldnedr analyses tgChareemmaitfsoiceralsvtteioosneoadnnutdchaetetopics of tsuWounisdpdteraeorinnsvitaniadbgneltedmhicneoogirrneosfusymsptetmionic Target the app to
sustainable consumption. narratives and solutions. specific user groups,
recognizing their needs,
Risk identification and Revisit the app’s value level of knowledge, and
management plans. proposition. reaction to gamification
        </p>
        <p>
          Besides visiting the store Keep an open channel
ratings and users’ and reply / act within a Acknowledge and thank
comments, the app can reasonable time, even if all the feedback provided
include an email address to it’s to note that acting – even when irrelevant
contact or even an in-app will take longer than to the app.
experience rating sheet. expected.
efforts among societal stakeholders and distributing
both the agency and responsibility to enable
sustainable consumption practices. To align with the
dimension of anticipation or envisioning future
research or design from current dynamics [
          <xref ref-type="bibr" rid="ref23">23</xref>
          ] SCA
creators should consider their apps’ unintended and
potentially harmful results. The SGD fully embraces
this dimension at the beginning of the process, asking
designers to review their values and ethics as their
first step, thus directing the visioning and exploration
of technological platforms according to the creators’
values and the users’ needs. Although some creators
already have risk management plans in place, some
struggle to identify these risk areas when it comes to
implementing gamification; in many instances, this is
due to their limited knowledge about gamification.
        </p>
        <p>
          The dimension of reflexivity relates to the values
and beliefs of the actors involved in science, public
collaboration, and dialogue [
          <xref ref-type="bibr" rid="ref23">23</xref>
          ]; it is needed for
creative problem-solving and engaging other
stakeholders in the design process. Almost all
interviewees claimed their apps were inspired by their
own needs and those of their immediate circles,
involving their users in the prototyping stages – one
app was even developed in a living lab environment –
incorporating their feedback and creating learning
loops to meet their users’ needs besides keeping
consistent with their values and objectives. Besides
being a practice of reflexivity, creating these feedback
loops is also a contribution to the inclusion dimension,
which is about identifying socially desirable outcomes
from the perspective of all stakeholders involved. An
all-encompassing take on this dimension may help SCA
creators to strengthen their apps not only through
feedback but also with codes of conduct clearly
presented to their users and partners, overcoming
mismatching expectations, such as expecting apps to
be entirely free of use even though the sustenance of
the creator may depend on the app. This dimension
also attains collaboration with experts and individuals
from other industries and sectors. All RRI dimensions
are interconnected, and responsiveness is a natural
companion to inclusion. While mainly related to the
mid and long-term risks of new technologies, this
dimension is also about transparency and accessibility
so that all stakeholders are aware that gamification is
a long-term investment that requires maintenance and
updates. Reporting on the app’s performance and
presenting the creators’ code of ethics encourage users
and potential partners to see their affinity with the app
and their eventual uptake. [
          <xref ref-type="bibr" rid="ref23">23</xref>
          ] also elaborate on the
dimension of care, or the human capacity to decide and
judge where individuals take responsibility for
decisions and actions carried out on their behalf. Care
is differentiated from inclusion because it looks into
the human as an inner decider who does not want to
be judged yet needs support to act, as noted by some
of the SCA.
        </p>
        <p>Error! Reference source not found. presents an o
verview of how SCA creators can embed RRI
dimensions into their design and management
processes. The products enlisted are part of the ICT
creation ecosystem facilitating RRI practices.</p>
      </sec>
    </sec>
    <sec id="sec-5">
      <title>5. Discussion</title>
      <p>
        This study set to explore the main concerns of SCA
creators when implementing gamification as an
innovation that investigates, induces, and intervenes
in consumption practices, depicting how RRI
dimensions could be integrated into the design and
implementation processes of these apps; thus,
contributing to innovation in SHCI and SCHI as a
research field in general. To this end, the analysis
departed from the higher understanding that ICT is
“any high-level socio-technical system that has the
potential to significantly affect the way humans
interact with the world” ([
        <xref ref-type="bibr" rid="ref26">26</xref>
        ], p. 204); therefore, the
creators of ICT solutions should be able to understand
the capabilities and constraints of technologies as they
emerge, as this helps to foresee what could be done
today to ensure that the social and ethical
consequences of technologies are not detrimental to
humans or the environment. Part of this
understanding conveys knowledge of what their users
consider a risk besides what they need. While the apps
may have very specific target users (e.g. the
communities where they operate, or the employees of
their corporate partners) in general, they all aim at
engaging users willing to live more sustainably,
supporting their efforts to do so. Therefore, we used
the results of [
        <xref ref-type="bibr" rid="ref30">30</xref>
        ] to align the creators’ considerations
and users’ requests to guide the discussion about the
elements of concern to gamify SCA responsibly
through the RRI dimensions.
      </p>
      <p>5.1. Users’ requests and perceived
risks</p>
      <sec id="sec-5-1">
        <title>Users’ request 1 – Monetary aspects</title>
        <p>
          Price and value for money. This area represents the
widest gap between the users’ requests and the
creators’ concerns. [
          <xref ref-type="bibr" rid="ref30">30</xref>
          ] elaborate on how the cost of
an app influences its perception and attractiveness for
downloading. In the case of SCA, the creators
expressed the presence of a pervasive notion that
sustainability-related apps for individual use should
be free, leaving the creators needing other
incomegeneration means to keep the app working and
relevant. While gamified apps seem to be more
attractive and last longer than their non-gamified
counterparts [
          <xref ref-type="bibr" rid="ref31">31</xref>
          ], gamification comes with a higher
price tag, as the creators need to keep up with the costs
behind the strategy and its implementation, as well as
come up with coherent gamification strategies,
hopefully steering clear from physical rewards that
convey additional charges. In general, balancing the
costs with the users’ expectations attains all the value
destroyers, as a botched gamification strategy might as
well lead to the app's disappearance. Enabling
inclusion and responsiveness practices in the design
process has helped the creators to identify their users’
needs, and it should also be a channel for the users to
understand the creators’ plights for maintaining the
apps and delivering the value they are looking for. All
the accounts of the five app creators whose apps
disappeared pointed to financial woes as one of the
main reasons. The creators behind the five apps that
switched business models from servicing individual
consumers to partnering with businesses expressed a
similar situation for their choices to keep their app
alive, even if that entailed curtailing their work for the
free versions of their apps and reaching narrower
audiences. Thus, enabling user awareness about the
actual costs of the apps should be part of the creators’
responsibility, instilling the dimension of care and
advancing RRI awareness and implementation
processes. It should also be part of the users’
responsibility. Part of this narrative should emphasize
how shifting consumption practices today may seem
costly, but the price to pay tomorrow is even higher if
remaining inactive.
        </p>
      </sec>
      <sec id="sec-5-2">
        <title>Users’ request 2 – Ease of use</title>
        <p>This rubric comprises notions related to the apps’
efficiency, attractiveness, usability, learnability, and
comprehensibility. Elements that gamification can
contribute to bringing forward if adequately
implemented. For the creators, these notions are the
core of their value proposition and, sometimes, the
reason not to team up with third-party advertisers, for
example. Recognizing that their apps are mainly used
on phones with limited screen space, seven creators
elaborated on the importance of maximizing the
screen space with relevant information and features
rather than adding unnecessary noise that may deter
users from exploring the app or staying loyal to it.
Considering the users’ experience is an example of
both anticipation and reflexivity, as the creators find a
way to communicate the intention of their apps in a
more straightforward and transparent manner than
under layers of text or hidden functions. The value
destroyers of human agency loss and homogenization
are the ones most likely to emerge if the app creators
do not pay the same attention to these user requests.
There is no one-fits-all approach to gamifying SCA;
thus, the creators should take enough time to plan the
gamification project before proceeding with the rest of
the design process [46], mainly if they will involve
other stakeholders as the RRI dimensions of inclusion
and reflexivity require.</p>
        <p>Users’ request 3 and number 1 perceivable risk –</p>
      </sec>
      <sec id="sec-5-3">
        <title>Safety aspects: privacy and security</title>
        <p>The leaky containers and coercive participation value
destroyers are the most likely to emerge if the creators
do not have clear guidelines for safely collecting and
managing their users’ data. The RRI responsiveness
dimension suggests creating a transparent code of
conduct and enabling transparent accountability
mechanisms. These practices attain RRI awareness
and implementation and to the assessment activities,
as all stakeholders have access to the app’s creators'
code of conduct, not only to their visions and value
statements as organized entities. The convergence of
risk perception and requests from the users’ side is a
clear indicator for the creators about what areas
should be prioritized and clearly communicated.
Moreover, developing performance indicators in this
regard can help strengthen their presence and
confidence among potential partners. While some of
the interviewed creators had comprehensive risk
plans, the reliance on the app store’s safeguards may
not be enough guarantee for users; therefore, the
creative teams behind the apps should plan strategic
sessions to consolidate their risk assessment and
management plans, keeping them flexible enough to
adapt to the development of technology, changes in
regulations, and changes in the market, thus enhancing
the dimension of anticipation too.</p>
      </sec>
      <sec id="sec-5-4">
        <title>User perceived risks 2, 3 and 4 - user-friendliness, pleasure of using the app and accuracy</title>
        <p>
          Here, the value destroyers illusion of change and
shallowness indicate a dangerous zone for the app
creators if they do not manage to provide gamification
elements consistent and coherent with what the app
stands for. As the app analysis [
          <xref ref-type="bibr" rid="ref31">31</xref>
          ] revealed, many
apps with high volumes of downloads and reviews rely
on out-of-app rewards in the shape of discount
vouchers that can be used for more consumption.
While this mechanism is presented as a “win-win”
situation for the user and the planet, this gamification
system leads to using the app for immediate, personal
gratification instead of adopting sustainable
consumption habits. Such situations directly confront
the RRI dimension of care, as the users’ trust in
someone making decisions for them, in this case, the
app is misleading the intention to consume better with
fewer resources.
        </p>
        <p>The perceived risk of accuracy is also a milestone
for the design of SCA. 12 creators noted how much of
their resources were invested in research and the
provision of reliable data. The apps created to provide
information for choice-making have several guidelines
for acquiring and curating information before it is
made available in the app. The creators are well aware
of the damage that providing false information would
convey and often rely on the feedback of their users to
improve their content. This activity is part of the RRI
domain of inclusion and reflexivity; as the creators
check if the data is correct, reliable, and relevant to
what the app stands for before including it as part of
their content. Thus, the gamification elements should
be able to reinforce the understanding that the
information presented is accurate.</p>
        <p>Although the germinal processes, resources, and
motivations to embark on an SCA creative journey
vary, the people behind these apps can consider the
recommendations below as an opportunity to
contribute to practices of sustainable
humancomputer interaction responsibly.</p>
        <p>1. Define their sustainability values and
ethical stances as a prerequisite for topical,
marketing research, and decision-making by:
• Developing a unified description of their
sustainability values and using it for envisioning
risks to create strategies to manage these (e.g.,
coming up with potential unintended negative
impacts they can mitigate). Reaching out to
potential users and investors can help define their
sustainability stances and draft solid risk
management plans;
• Educating themselves on sustainability and
gamification topics to find suitable strategies and
include relevant in-app content (e.g., learning
about the true costs of developing gamified apps;
and avoiding rebound effects based on physical
rewards);
• Conducting appropriate market and user
analysis (e.g., for tailoring gamification aspects and
in-app content to the right audience).
2. Enable creative problem-solving through
participatory design:
• Involving users and all other relevant
stakeholders throughout the development process
(e.g., including diverse perspectives for improved
app usability);
• Considering the users’ opinions and
experience by asking and addressing feedback in
all stages of the development and use (e.g., for
providing relevant and accurate content only, for
transparent and straightforward communication
of app intention).
3. Implement holistic approaches and
crosscultural cooperation for socially desirable
outcomes:
• Elaborating a code of conduct based on the
defined sustainability values and ethical stances;
• Deploying cross-sectoral collaboration and
involving experts of different sectors adhering to
the code of conduct, for a better and more relevant
representation of the topic; for enhanced adoption
of sustainable consumption behaviors; and, for
developing a holistic approach yielding
generalizable socially desirable outcomes;
• Creating and managing realistic expectations
of all parties (e.g., financial management of the
app);
• Enabling transparency in processes and
safety guidelines/codes of conduct (e.g.,
transparent communication that delivers value;
public information about app’s values; and
transparency in data privacy and security).
4. Develop and maintain responsiveness
strategies for transparent communication and
accessible solutions:
• Elaborating practical and accessible
communication channels (e.g., for making the
process of leaving feedback appealing to the user,
thus contributing to a more desirable app);
• Implementing effective reporting
mechanisms, including report validation (e.g., for
establishing and nurturing regular and transparent
communication with the stakeholders; for
ensuring correct and digestible reports);
• Considering and addressing user feedback in
a prompt manner (e.g., for maintaining users’
engagement and development of user-friendly and
accessible solutions).
5. Nurturing a sense of shared responsibility
between app creators and users:
• Make the code of conduct, sustainability
values, and ethical stances public to the users;
• Building and maintaining relationships with
users and other stakeholders through shared
efforts (e.g. for long-lasting partnerships; for
avoiding human agency loss among users; and for
achieving common sustainability goals).
The above discussion points illustrate how this study
contributes to sustainable human-computer
interaction by providing SCA creators with
recommendations to support their apps’ objectives
while bearing unintended consequences in mind,
particularly if applying gamification. As most of the
creators provided accounts highlighting their focus on
the care dimension, since their apps are enablers of
personal change, they need to reflect on the messages
they are conveying through their apps, and the
responsibility of offering a tool for individual behavior
change brings about. This paper contributes to the
studies of SHCI through design, presenting how
integrating RRI dimensions into the design of gamified
sustainable consumption apps may help advancing
digital societies from a value-based approach to
innovation. The postulates of this study also entail
many limitations, like the size of the sample. Though
sustainable consumption apps are a very specific
niche, the results presented are based on the
qualitative analysis of the accounts of 21 individuals.
Also, none of the analytical frameworks used was
purposely designed for mobile apps, although both
relate to the study’s subject; the 4P’s are
considerations for RRI in the ICT sector, while the SGD
is about value-based gamification design. Perhaps
developing our own framework with the themes that
emerged from the data, or using other frameworks
specifically created for SCA would yield additional
results; these options were not explored due to the size
of the sample and the time restrictions inherent to this
study. Another existing issue is the applicability of the
recommendations hereby presented. Although on
paper, they seem straightforward, in practice, SCA
creators may encounter many obstacles to
implementing them, from the time available to carry
out these processes to the costs of maintaining any
additional features or involving more people in
activities beyond their job descriptions. These
recommendations intend to add value to the SCA, and
the creators are invited to consider the ones that
represent a low-hanging fruit according to their
current reality. Another limitation comes in the shape
of the users’ requests and perceived risks, as the
analysis was done referring to existing academic
research instead of interviews with the users of the
apps, which could help depicting more specific request
for the creators to heed. After reviewing similar
studies, the chosen one represented the most
comprehensive and updated overview of users’ needs.
Including other similar studies may help to provide
more robust findings. Nonetheless, the novelty of this
research lies in its value-based approach to identify
and embed the app creators’ considerations for
ethically applying gamification within the RRI
dimensions, helping to narrow the gap between
studies focusing on app design from the user
experience and those seeking examples of RRI in
practice.</p>
      </sec>
    </sec>
    <sec id="sec-6">
      <title>6. Conclusion</title>
      <p>As the use of apps for everyday activities keeps
growing, so does the studying of how apps function
and what users require so that app creators can
operate and survive in an increasingly competitive
field. When it comes to sustainability by and through
design, ICT solutions have a strong orientation toward
consumption and production practices. This study
focuses on the consumption aspect, as the narratives
related to it vary enormously, and so do the apps
created with the intention of enabling behaviors that
contribute to it. While existing research corroborates
the inclusion of gamification as a part of the design
efforts to promote the efficient use of resources and
positively impact the environment, there is less
information about what considerations the creators
have about the unintended effects that their apps may
have. This study emphasizes sustainability through
design by proposing a values-based approach to
designing sustainable consumption apps, recognizing
gamification as an innovation that reveals problems,
stimulates novel behaviors, and aims at transforming
processes for improved effectiveness and engagement.
In addition, this study highlights that utilizing
gamification as an innovation strategy in SHCI, not
only helps to address many of the shortcomings of
ecofeedback systems but could result in both socially
desirable and undesirable consequences, drawing
attention of SCA creators to potential risks and calling
for their preparedness to prevent or mitigate them as
part of their apps' value propositions.</p>
    </sec>
    <sec id="sec-7">
      <title>Acknowledgements</title>
      <p>The authors would like to wholeheartedly thank the 21
app creators that shared their journeys with us. This
study was supported by the Foundation for Economic
Education [Grant No. 210301 - GAMETH]; the
Academy of Finland Flagship Program [Grant No.
337653 - Forest-Human-Machine Interplay (UNITE)];
and the Fortum and Neste Foundation [grant
20200029].</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          [1]
          <string-name>
            <given-names>G.</given-names>
            <surname>Brundtland</surname>
          </string-name>
          ,
          <source>Report of the World Commission on Environment and Development: Our Common Future, United Nations General Assembly document A/42/427</source>
          ,
          <year>1987</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          [2]
          <string-name>
            <given-names>S. M.</given-names>
            <surname>Geiger</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Fischer</surname>
          </string-name>
          ,
          <string-name>
            <given-names>U.</given-names>
            <surname>Schrader</surname>
          </string-name>
          ,
          <article-title>Measuring What Matters in Sustainable Consumption: An Integrative Framework for the Selection of Relevant Behaviors</article-title>
          ,
          <source>Sustainable Development</source>
          ,
          <volume>26</volume>
          .1 (
          <year>2018</year>
          )
          <fpage>18</fpage>
          -
          <lpage>33</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          [3]
          <string-name>
            <given-names>J.</given-names>
            <surname>Mankoff</surname>
          </string-name>
          ,
          <string-name>
            <given-names>E.</given-names>
            <surname>Blevis</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Borning</surname>
          </string-name>
          ,
          <string-name>
            <given-names>B.</given-names>
            <surname>Friedman</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S. R.</given-names>
            <surname>Fussell</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Hasbrouck</surname>
          </string-name>
          ,
          <string-name>
            <given-names>P.</given-names>
            <surname>Sengers</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Woodruff</surname>
          </string-name>
          ,
          <article-title>Environmental sustainability and interaction</article-title>
          ,
          <source>CHI '07 Extended Abstracts on Human Factors in Computing Systems</source>
          ,
          <year>2007</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          [4]
          <string-name>
            <given-names>V.</given-names>
            <surname>P. De Almeida Neris</surname>
          </string-name>
          ,
          <string-name>
            <given-names>K.</given-names>
            da
            <surname>Hora</surname>
          </string-name>
          <string-name>
            <surname>Rodrigues</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R. F.</given-names>
            <surname>Lima</surname>
          </string-name>
          ,
          <article-title>A Systematic Review of Sustainability and Aspects of Human-Computer Interaction</article-title>
          , in: M. Kurosu (Eds)
          <article-title>Human-Computer Interaction</article-title>
          . Applications and Services,
          <source>HCI 2014, Lecture Notes in Computer Science</source>
          , vol
          <volume>8512</volume>
          , Springer, Cham,
          <year>2014</year>
          . https://doi.org/10.1007/978-3-
          <fpage>319</fpage>
          -07227-2_
          <fpage>71</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          [5]
          <string-name>
            <given-names>C.</given-names>
            <surname>Remy</surname>
          </string-name>
          ,
          <string-name>
            <given-names>O.</given-names>
            <surname>Bates</surname>
          </string-name>
          ,
          <string-name>
            <given-names>V.</given-names>
            <surname>Thomas</surname>
          </string-name>
          ,
          <string-name>
            <surname>E. Huang,</surname>
          </string-name>
          <article-title>The Limits of Evaluating Sustainability</article-title>
          ,
          <source>in: Proceedings of the 2017 Workshop on Computing Within Limits (LIMITS '17)</source>
          , Association for Computing Machinery, New York, NY, USA,
          <year>2017</year>
          , pp.
          <fpage>103</fpage>
          -
          <lpage>110</lpage>
          . https://doi.org/10.1145/3080556.3080567.
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          [6]
          <string-name>
            <given-names>S.</given-names>
            <surname>Scuri</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Ferreira</surname>
          </string-name>
          ,
          <string-name>
            <given-names>N.</given-names>
            <surname>Nunes</surname>
          </string-name>
          ,
          <string-name>
            <given-names>V.</given-names>
            <surname>Nisi</surname>
          </string-name>
          ,
          <string-name>
            <surname>C.</surname>
          </string-name>
          <article-title>Mulligan, Hitting the Triple Bottom Line: Widening the HCI Approach to Sustainability</article-title>
          , in
          <source>: Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems (CHI '22)</source>
          , Association for Computing Machinery, New York, NY, USA, Article
          <volume>332</volume>
          ,
          <year>2022</year>
          , pp.
          <fpage>1</fpage>
          -
          <lpage>19</lpage>
          . https://doi.org/10.1145/3491102.3517518.
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          [7]
          <string-name>
            <given-names>L.</given-names>
            <surname>Hansson</surname>
          </string-name>
          ,
          <string-name>
            <given-names>T. Cerratto</given-names>
            <surname>Pargman</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Pargman</surname>
          </string-name>
          ,
          <article-title>A Decade of Sustainable HCI: Connecting SHCI to the Sustainable Development Goals</article-title>
          ,
          <source>in: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (CHI '21)</source>
          , Association for Computing Machinery, New York, NY, USA, Article
          <volume>300</volume>
          ,
          <year>2021</year>
          , pp.
          <fpage>1</fpage>
          -
          <lpage>19</lpage>
          . https://doi.org/10.1145/3411764.3445069.
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          [8]
          <string-name>
            <given-names>C.</given-names>
            <surname>Remy</surname>
          </string-name>
          ,
          <string-name>
            <given-names>O.</given-names>
            <surname>Bates</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Dix</surname>
          </string-name>
          , V. Thomas,
          <string-name>
            <given-names>M.</given-names>
            <surname>Hazas</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Friday</surname>
          </string-name>
          ,
          <string-name>
            <given-names>E. M.</given-names>
            <surname>Huang</surname>
          </string-name>
          .
          <source>Evaluation Beyond Usability: Validating Sustainable HCI Research, in: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI '18)</source>
          , Association for Computing Machinery, New York, NY, USA,
          <year>2018</year>
          , Paper 216, pp.
          <fpage>1</fpage>
          -
          <lpage>14</lpage>
          , https://doi.org/10.1145/3173574.3173790.
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          [9]
          <string-name>
            <given-names>F.</given-names>
            <surname>Cellina</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Bucher</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J. Veiga</given-names>
            <surname>Simão</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R.</given-names>
            <surname>Rudel</surname>
          </string-name>
          , and
          <string-name>
            <given-names>M.</given-names>
            <surname>Raubal</surname>
          </string-name>
          ,
          <article-title>Beyond Limitations of Current Behaviour Change Apps for Sustainable Mobility: Insights from a User-Centered Design</article-title>
          and
          <string-name>
            <given-names>Evaluation</given-names>
            <surname>Process</surname>
          </string-name>
          , Sustainability,
          <volume>11</volume>
          .8 (
          <year>2019</year>
          )
          <article-title>2281</article-title>
          . https://doi.org/10.3390/su11082281.
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          [10]
          <string-name>
            <given-names>J.</given-names>
            <surname>Froehlich</surname>
          </string-name>
          ,
          <string-name>
            <given-names>L.</given-names>
            <surname>Findlater</surname>
          </string-name>
          ,
          <string-name>
            <surname>J. Landay,</surname>
          </string-name>
          <article-title>The design of eco-feedback technology</article-title>
          ,
          <source>in: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '10)</source>
          , Association for Computing Machinery, New York, NY, USA,
          <year>2010</year>
          , pp.
          <fpage>1999</fpage>
          -
          <lpage>2008</lpage>
          . https://doi.org/10.1145/1753326.1753629.
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          [11]
          <string-name>
            <given-names>J.</given-names>
            <surname>Hamari</surname>
          </string-name>
          , Gamification, in: G. Ritzer,
          <string-name>
            <surname>C.</surname>
          </string-name>
          Rojek (Eds.),
          <source>The Blackwell Encyclopedia of Sociology</source>
          , John Wiley &amp; Sons,
          <year>2019</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          [12]
          <string-name>
            <given-names>A.</given-names>
            <surname>Shahri</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Hosseini</surname>
          </string-name>
          ,
          <string-name>
            <given-names>K.</given-names>
            <surname>Phalp</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Taylor</surname>
          </string-name>
          , R. Ali,
          <article-title>Towards a Code of Ethics for Gamification at Enterprise</article-title>
          , in: U. Frank, P. Loucopoulos, Ó. Pastor, I. Petrounias (Eds.)
          <article-title>The Practice of Enterprise Modeling</article-title>
          ,
          <source>PoEM 2014, Lecture Notes in Business Information Processing</source>
          , vol
          <volume>197</volume>
          , Springer, Berlin, Heidelberg,
          <year>2014</year>
          . https://doi.org/10.1007/978-3-
          <fpage>662</fpage>
          -45501- 2_
          <fpage>17</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          [13]
          <string-name>
            <given-names>D.</given-names>
            <surname>Stroud</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C.</given-names>
            <surname>Evans</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Weinel</surname>
          </string-name>
          ,
          <article-title>Innovating for energy efficiency: Digital gamification in the European steel industry</article-title>
          ,
          <source>European Journal of Industrial Relations</source>
          ,
          <volume>26</volume>
          .4 (
          <year>2020</year>
          )
          <fpage>419</fpage>
          -
          <lpage>437</lpage>
          . https://doi.org/10.1177/0959680120951707.
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          [14]
          <string-name>
            <given-names>R.</given-names>
            <surname>Mulcahy</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R.</given-names>
            <surname>Russell-Bennett</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Iacobucci</surname>
          </string-name>
          ,
          <article-title>Designing gamified apps for sustainable consumption: A field study</article-title>
          ,
          <source>Journal of Business Research</source>
          ,
          <volume>106</volume>
          (
          <year>2020</year>
          )
          <fpage>377</fpage>
          -
          <lpage>387</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref15">
        <mixed-citation>
          [15]
          <string-name>
            <given-names>B. D.</given-names>
            <surname>Douglas</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Brauer</surname>
          </string-name>
          ,
          <article-title>Gamification to prevent climate change: a review of games and apps for sustainability</article-title>
          ,
          <source>Current Opinion in Psychology</source>
          ,
          <volume>42</volume>
          (
          <year>2021</year>
          )
          <fpage>89</fpage>
          -
          <lpage>94</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref16">
        <mixed-citation>
          [16]
          <string-name>
            <given-names>M.</given-names>
            <surname>Jirotka</surname>
          </string-name>
          ,
          <string-name>
            <given-names>B.</given-names>
            <surname>Grimpe</surname>
          </string-name>
          ,
          <string-name>
            <given-names>B.</given-names>
            <surname>Stahl</surname>
          </string-name>
          , G. Eden,
          <string-name>
            <given-names>M.</given-names>
            <surname>Hartswood</surname>
          </string-name>
          ,
          <article-title>Responsible research and innovation in the digital age</article-title>
          ,
          <source>in: Communications of the ACM</source>
          ,
          <volume>60</volume>
          .5 (
          <issue>2017</issue>
          )
          <fpage>2</fpage>
          -
          <lpage>68</lpage>
          . https://doi.org/10.1145/3064940.
        </mixed-citation>
      </ref>
      <ref id="ref17">
        <mixed-citation>
          [17]
          <string-name>
            <given-names>M.</given-names>
            <surname>Schuijff</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A. M.</given-names>
            <surname>Dijkstra</surname>
          </string-name>
          , Practices of Responsible Research and Innovation: A Review,
          <source>Science and engineering ethics</source>
          ,
          <volume>26</volume>
          .2 (
          <year>2020</year>
          )
          <fpage>533</fpage>
          -
          <lpage>574</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref18">
        <mixed-citation>
          [18]
          <string-name>
            <given-names>A.</given-names>
            <surname>Shpakova</surname>
          </string-name>
          ,
          <string-name>
            <given-names>V.</given-names>
            <surname>Dörfler</surname>
          </string-name>
          ,
          <string-name>
            <surname>J. MacBryde</surname>
          </string-name>
          , Gamifying Innovation and
          <article-title>Innovating Through Gamification</article-title>
          , in: V.
          <string-name>
            <surname>Ratten</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          <string-name>
            <surname>Jones</surname>
            ,
            <given-names>V.</given-names>
          </string-name>
          <string-name>
            <surname>Braga</surname>
          </string-name>
          , C.S. Marques (Eds.) Subsistence Entrepreneurship, Contributions to Management Science, Springer, Cham,
          <year>2019</year>
          . https://doi.org/10.1007/978-3-
          <fpage>030</fpage>
          -11542-5_
          <fpage>10</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref19">
        <mixed-citation>
          [19]
          <string-name>
            <surname>F. M. AlSaad</surname>
          </string-name>
          ,
          <string-name>
            <surname>C. M. Durugbo</surname>
          </string-name>
          , Gamification-as
          <string-name>
            <surname>Innovation</surname>
          </string-name>
          : A Review,
          <source>International Journal of Innovation and Technology Management</source>
          ,
          <volume>18</volume>
          .5 (
          <year>2021</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref20">
        <mixed-citation>
          [20]
          <string-name>
            <given-names>D.</given-names>
            <surname>Berdichevsky</surname>
          </string-name>
          , E. Neuenschwander,
          <article-title>Toward an ethics of persuasive technology</article-title>
          ,
          <source>in: Communications of the ACM</source>
          ,
          <volume>42</volume>
          .5 (
          <year>1999</year>
          )
          <fpage>51</fpage>
          -
          <lpage>58</lpage>
          . https://doi.org/10.1145/301353.301410.
        </mixed-citation>
      </ref>
      <ref id="ref21">
        <mixed-citation>
          [21]
          <string-name>
            <given-names>R.</given-names>
            <surname>Owen</surname>
          </string-name>
          , R. von Schomberg,
          <string-name>
            <given-names>P.</given-names>
            <surname>Macnaghten</surname>
          </string-name>
          ,
          <article-title>An unfinished journey? Reflections on a decade of responsible research and innovation</article-title>
          ,
          <source>Journal of Responsible Innovation</source>
          ,
          <volume>8</volume>
          .2 (
          <year>2021</year>
          )
          <fpage>217</fpage>
          -
          <lpage>233</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref22">
        <mixed-citation>
          [22]
          <string-name>
            <given-names>R.</given-names>
            <surname>von Schomberg</surname>
          </string-name>
          ,
          <article-title>A vision of responsible innovation</article-title>
          . In: R. Owen,
          <string-name>
            <given-names>M.</given-names>
            <surname>Heintz</surname>
          </string-name>
          and J.
          <string-name>
            <surname>Bessant</surname>
          </string-name>
          (Eds.) Responsible Innovation, John Wiley,
          <year>2013</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref23">
        <mixed-citation>
          [23]
          <string-name>
            <given-names>M.</given-names>
            <surname>Burget</surname>
          </string-name>
          , E. Bardone,
          <string-name>
            <given-names>M.</given-names>
            <surname>Pedaste</surname>
          </string-name>
          ,
          <article-title>Definitions and Conceptual Dimensions of Responsible Research and Innovation: A Literature Review</article-title>
          ,
          <source>Science and Engineering Ethics</source>
          ,
          <volume>23</volume>
          .1 (
          <issue>2016</issue>
          )
          <fpage>1</fpage>
          -
          <lpage>19</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref24">
        <mixed-citation>
          [24]
          <string-name>
            <surname>I. van de Poel</surname>
            , L. Asveld,
            <given-names>S.</given-names>
          </string-name>
          <string-name>
            <surname>Flipse</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          <string-name>
            <surname>Klaassen</surname>
            ,
            <given-names>V.</given-names>
          </string-name>
          <string-name>
            <surname>Scholten</surname>
          </string-name>
          , E. Yaghmaei, Company Strategies for Responsible Research and
          <string-name>
            <surname>Innovation (RRI): A Conceptual Model</surname>
          </string-name>
          , Sustainability,
          <volume>9</volume>
          .11 (
          <year>2017</year>
          )
          <year>2045</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref25">
        <mixed-citation>
          [25]
          <string-name>
            <given-names>A.</given-names>
            <surname>Porcari</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Pimponi</surname>
          </string-name>
          , E. Borsella,
          <string-name>
            <given-names>P.</given-names>
            <surname>Klaassen</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M. J.</given-names>
            <surname>Maia</surname>
          </string-name>
          , E. Mantovani,
          <article-title>Supporting RRI uptake in industry: A qualitative and multi-criteria approach to analysing the costs and benefits of implementation</article-title>
          , in E. Yaghmaei, I. van de Poel (Eds.) Assessment of Responsible Innovation, Routledge,
          <year>2020</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref26">
        <mixed-citation>
          [26]
          <string-name>
            <given-names>B. C.</given-names>
            <surname>Stahl</surname>
          </string-name>
          , G. Eden,
          <string-name>
            <given-names>M.</given-names>
            <surname>Jirotka</surname>
          </string-name>
          , Responsible Research and
          <article-title>Innovation in Information and Communication Technology: Identifying and Engaging with the Ethical Implications of ICTs</article-title>
          , in: R.
          <string-name>
            <surname>Owen</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          <string-name>
            <surname>Bessant</surname>
          </string-name>
          , M. Heintz (Eds.) Responsible Innovation:
          <article-title>Managing the Responsible Emergence of Science and</article-title>
          Innovation in Society, John Wiley &amp; Sons,
          <year>2013</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref27">
        <mixed-citation>
          [27]
          <string-name>
            <given-names>J.</given-names>
            <surname>Stilgoe</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Guston</surname>
          </string-name>
          , Responsible research and innovation, in: U. Felt,
          <string-name>
            <given-names>R.</given-names>
            <surname>Fouche</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C. A.</given-names>
            <surname>Miller</surname>
          </string-name>
          ,
          <string-name>
            <given-names>L.</given-names>
            <surname>Smith-Doerr</surname>
          </string-name>
          (Eds.),
          <source>The Handbook of Science and Technology Studies, fourth edition</source>
          , MIT Press, Cambridge, MA, USA,
          <year>2016</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref28">
        <mixed-citation>
          [28]
          <string-name>
            <given-names>E.</given-names>
            <surname>Yaghmaei</surname>
          </string-name>
          ,
          <article-title>Responsible research and innovation key performance indicators in industry: A case study in the ICT domain</article-title>
          ,
          <source>Journal of Information, Communication and Ethics in Society, 16.2</source>
          (
          <year>2018</year>
          )
          <fpage>214</fpage>
          -
          <lpage>234</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref29">
        <mixed-citation>
          [29]
          <string-name>
            <given-names>M.</given-names>
            <surname>Raftopoulos</surname>
          </string-name>
          ,
          <article-title>Towards gamification transparency: A conceptual framework for the development of responsible gamified enterprise systems</article-title>
          ,
          <source>Journal of Gaming &amp; Virtual Worlds</source>
          ,
          <volume>6</volume>
          .2 (
          <year>2014</year>
          )
          <fpage>159</fpage>
          -
          <lpage>178</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref30">
        <mixed-citation>
          [30]
          <string-name>
            <given-names>T.</given-names>
            <surname>Hunger</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Arnold</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R.</given-names>
            <surname>Pestinger</surname>
          </string-name>
          ,
          <article-title>Risks and Requirements in Sustainable App Development -</article-title>
          A
          <string-name>
            <surname>Review</surname>
          </string-name>
          ,
          <year>Sustainability</year>
          ,
          <volume>15</volume>
          .8 (
          <year>2023</year>
          )
          <fpage>7018</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref31">
        <mixed-citation>
          [31]
          <string-name>
            <given-names>G.</given-names>
            <surname>Guillen Mandujano</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D. Fernandez</given-names>
            <surname>Galeote</surname>
          </string-name>
          ,
          <string-name>
            <given-names>N.</given-names>
            <surname>Sicevic</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Hamari</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Quist</surname>
          </string-name>
          ,
          <article-title>Gamified apps for sustainable consumption: a systematic review</article-title>
          ,
          <source>in: Proceedings of the 6th International GamiFIN Conference</source>
          , v.
          <volume>3147</volume>
          ,
          <year>2022</year>
          , pp.
          <fpage>135</fpage>
          -
          <lpage>145</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref32">
        <mixed-citation>
          [32]
          <string-name>
            <surname>United</surname>
            <given-names>Nations</given-names>
          </string-name>
          ,
          <source>Transforming our world: The 2030 Agenda for Sustainable Development</source>
          ,
          <year>2015</year>
          . URL: https://sdgs.un.
          <source>org/2030agenda. Last accessed: November</source>
          <volume>17</volume>
          ,
          <year>2023</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref33">
        <mixed-citation>
          [33]
          <string-name>
            <given-names>T. P. L.</given-names>
            <surname>Nghiem</surname>
          </string-name>
          ,
          <string-name>
            <given-names>L. R.</given-names>
            <surname>Carrasco</surname>
          </string-name>
          ,
          <article-title>Mobile Applications to Link Sustainable Consumption with Impacts on the Environment and Biodiversity</article-title>
          , BioScience,
          <volume>66</volume>
          .5 (
          <year>2016</year>
          )
          <fpage>384</fpage>
          -
          <lpage>392</lpage>
          . https://doi.org/10.1093/biosci/biw016.
        </mixed-citation>
      </ref>
      <ref id="ref34">
        <mixed-citation>
          [34]
          <string-name>
            <given-names>G. M.</given-names>
            <surname>Guillen</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Hamari</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Quist</surname>
          </string-name>
          ,
          <article-title>Gamification of sustainable consumption: A systematic literature review</article-title>
          ,
          <source>in: Proceedings of the 54th Hawaii International Conference on System Sciences</source>
          ,
          <year>2021</year>
          , pp.
          <fpage>1345</fpage>
          -
          <lpage>1354</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref35">
        <mixed-citation>
          [35]
          <string-name>
            <given-names>P. S.</given-names>
            <surname>Shukla</surname>
          </string-name>
          ,
          <string-name>
            <given-names>P. V.</given-names>
            <surname>Nigam</surname>
          </string-name>
          ,
          <article-title>E-shopping using mobile apps and the emerging consumer in the digital age of retail hyper personalization: An insight</article-title>
          , Pacific Business Review International,
          <volume>10</volume>
          .10 (
          <issue>2018</issue>
          ) pp.
          <fpage>131</fpage>
          -
          <lpage>139</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref36">
        <mixed-citation>
          [36]
          <string-name>
            <given-names>N.</given-names>
            <surname>Arambepola</surname>
          </string-name>
          ,
          <string-name>
            <given-names>L.</given-names>
            <surname>Munasinghe</surname>
          </string-name>
          ,
          <article-title>A way forward for Sustainable Human-Computer Interaction</article-title>
          , in
          <source>: Proceedings of the International Conference on Applied and Pure Sciences (ICAPS 2021- Kelaniya)</source>
          , vol.
          <volume>2</volume>
          ,
          <issue>2022</issue>
          , p.
          <fpage>45</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref37">
        <mixed-citation>
          [37]
          <string-name>
            <given-names>M. A.</given-names>
            <surname>Tremblay</surname>
          </string-name>
          , The Key Informant Technique:
          <string-name>
            <given-names>A</given-names>
            <surname>Non-Ethnographic</surname>
          </string-name>
          <string-name>
            <surname>Application</surname>
          </string-name>
          , in: R. G. Burgess (Ed.),
          <source>Field Research, first published</source>
          <year>1982</year>
          , Routledge,
          <year>2003</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref38">
        <mixed-citation>
          [38]
          <string-name>
            <given-names>G.</given-names>
            <surname>Guillen</surname>
          </string-name>
          <string-name>
            <surname>Mandujano</surname>
          </string-name>
          ,
          <source>Key informant semistructured interview guideline</source>
          ,
          <year>2024</year>
          . https://osf.io/nhw7a
        </mixed-citation>
      </ref>
      <ref id="ref39">
        <mixed-citation>
          [39]
          <string-name>
            <given-names>T. W.</given-names>
            <surname>Kim</surname>
          </string-name>
          ,
          <string-name>
            <given-names>K.</given-names>
            <surname>Werbach</surname>
          </string-name>
          ,
          <article-title>More than just a game: ethical issues in gamification</article-title>
          ,
          <source>Ethics and Information Technology</source>
          ,
          <volume>18</volume>
          (
          <year>2016</year>
          )
          <fpage>157</fpage>
          -
          <lpage>173</lpage>
          . https://doi.org/10.1007/s10676-016-9401-5.
        </mixed-citation>
      </ref>
      <ref id="ref40">
        <mixed-citation>
          [40]
          <string-name>
            <given-names>R.</given-names>
            <surname>Mulcahy</surname>
          </string-name>
          ,
          <string-name>
            <given-names>K.</given-names>
            <surname>Letheren</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R.</given-names>
            <surname>McAndrew</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C.</given-names>
            <surname>Glavas</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R.</given-names>
            <surname>Russell-Bennett</surname>
          </string-name>
          ,
          <article-title>Are households ready to engage with smart home technology?</article-title>
          ,
          <source>Journal of Marketing Management</source>
          ,
          <volume>35</volume>
          .
          <fpage>15</fpage>
          -
          <lpage>16</lpage>
          (
          <year>2019</year>
          )
          <fpage>1370</fpage>
          -
          <lpage>1400</lpage>
          . Doi:
          <volume>10</volume>
          .1080/0267257X.
          <year>2019</year>
          .
          <volume>1680568</volume>
          .
        </mixed-citation>
      </ref>
      <ref id="ref41">
        <mixed-citation>
          [41]
          <string-name>
            <given-names>N.</given-names>
            <surname>Shevchuk</surname>
          </string-name>
          ,
          <string-name>
            <given-names>K.</given-names>
            <surname>Degirmenci</surname>
          </string-name>
          ,
          <string-name>
            <given-names>H.</given-names>
            <surname>Oinas-Kukkonen</surname>
          </string-name>
          ,
          <article-title>Adoption of gamified persuasive systems to encourage sustainable behaviors: interplay between perceived persuasiveness and cognitive absorption</article-title>
          ,
          <source>in: ICIS 2019 Proceedings, 3</source>
          ,
          <year>2019</year>
          .
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>