<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>Uncovering Sustainability Requirements: An Exploratory Case Study in Canteens</article-title>
      </title-group>
      <contrib-group>
        <aff id="aff0">
          <label>0</label>
          <institution>Martina Z. Huber, Lorenz M. Hilty, Martin Glinz Department of Informatics University of Zurich</institution>
          ,
          <country country="CH">Switzerland</country>
        </aff>
      </contrib-group>
      <abstract>
        <p>-Software systems are embedded into daily life and as such have significant effects on the behavior and decision making of both their users and the people affected by using these systems. Such effects can be positive or negative. Considering them in requirements engineering (RE) is an important step towards sustainable development, as RE strongly influences the development and the evolution of a software system. So far, RE researchers have focused on requirements about minimizing negative environmental effects. However, effects that are enabled by using a software system can also be positive. For example, a software system could motivate its users to take environment-friendly decisions. Corresponding requirements about such positive enabling effects have been far less addressed. In this paper, we present an exploratory case study where we elicited requirements about positive enabling effects with respect to environmental sustainability. The project we studied is about extending an existing decision support system for meal planning in canteens by game-based elements. The extended system shall motivate meal planners who work in canteens to make environment-friendly choices. Our contribution is an exemplar of concrete sustainability requirements as well as insight into the nature of sustainability requirements about positive effects that are enabled by the use of a software system. Index Terms-enabling effects; sustainability requirements; sustainable software systems; gamification; exploratory study;</p>
      </abstract>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>I. INTRODUCTION</title>
      <p>
        Any software system and its utilization have effects causing
environmental impacts [
        <xref ref-type="bibr" rid="ref1">1</xref>
        ], [
        <xref ref-type="bibr" rid="ref2">2</xref>
        ], [
        <xref ref-type="bibr" rid="ref3">3</xref>
        ]. We categorize these
effects into two groups. One group relates to the direct
effects of running a software system, e.g., the energy and
resources required for powering and cooling the computing
equipment the system runs on or connects to. Such effects are
usually considered to be negative with regard to sustainable
development. The second group relates to enabling effects.
      </p>
      <p>These are effects that are enabled by using a software system.</p>
      <p>
        Enabling effects can be the result of automated processes
and/or of human behavior that is influenced by using the
software system. With regard to sustainable development,
enabling effects can be both positive and negative [
        <xref ref-type="bibr" rid="ref4">4</xref>
        ]. For
example, heating management software can reduce the energy
consumption of a building significantly, which is positive.
      </p>
      <p>
        An e-commerce system providing business-to-consumer and
consumer-to-consumer sales services may influence human
behavior positively by encouraging people to sell used goods
so that they get re-used, but it can also lead to over-consumption,
which is negative. For a more detailed classification of effects
of software systems on sustainability see Berkhout &amp; Hertin
[
        <xref ref-type="bibr" rid="ref5">5</xref>
        ] or Hilty &amp; Aebischer [
        <xref ref-type="bibr" rid="ref4">4</xref>
        ].
      </p>
      <p>
        Our dependency on software systems [
        <xref ref-type="bibr" rid="ref6">6</xref>
        ] and their
ubiquity [
        <xref ref-type="bibr" rid="ref7">7</xref>
        ] within many societies has intensified the impact of
enabling effects. Consequently, when striving for sustainable
development, we must take into account potential enabling
effects when developing a software system, in order to avoid
negative effects and leverage positive ones.
      </p>
      <p>
        The importance of considering sustainable development in
software engineering has been emphasized by researchers
in the field [
        <xref ref-type="bibr" rid="ref6">6</xref>
        ], [
        <xref ref-type="bibr" rid="ref8">8</xref>
        ], [
        <xref ref-type="bibr" rid="ref9">9</xref>
        ]. Due to its early influence in the
development process of a software system, requirements
engineering (RE) is considered to have the biggest impact
on the eventual effects of a software system [
        <xref ref-type="bibr" rid="ref10">10</xref>
        ]. As such,
RE provides promising opportunities to affect the transition
towards sustainable development significantly [
        <xref ref-type="bibr" rid="ref2">2</xref>
        ], [
        <xref ref-type="bibr" rid="ref3">3</xref>
        ], [
        <xref ref-type="bibr" rid="ref10">10</xref>
        ],
[
        <xref ref-type="bibr" rid="ref6">6</xref>
        ], [
        <xref ref-type="bibr" rid="ref8">8</xref>
        ].
      </p>
      <p>
        Current RE contributions considering sustainable
development have focused on minimizing negative environmental
effects and referred to corresponding requirements as quality
requirements, e.g. [
        <xref ref-type="bibr" rid="ref3">3</xref>
        ], [
        <xref ref-type="bibr" rid="ref7">7</xref>
        ], [
        <xref ref-type="bibr" rid="ref1">1</xref>
        ], [
        <xref ref-type="bibr" rid="ref10">10</xref>
        ]. Sustainability requirements
related to positive effects however, have not received much
attention and can not be limited to quality aspects.
      </p>
      <p>Our goal is to better understand sustainability requirements
related to positive effects, specifically the ones that are enabled
by end-users while using a software system. This paper
describes an exploratory case study in which we elicit and
discuss such sustainability requirements for a decision support
system (DSS) that is extended with game-based elements for
motivating meal planners to make choices that reduce the CO2
value of their meals.</p>
      <p>The contribution of our study is twofold. Firstly, we present
an exemplar of concrete sustainability requirements related to
positive enabling effects. Secondly, we discuss these
sustainability requirements. The study reveals that when considering
positive enabling effects (i) corresponding requirements include
new requirements and existing requirements that become
more important, and (ii) treating these requirements as quality
requirements is inadequate as we found functional requirements
and constraints. The study further provides some indication
that (iii) corresponding requirements can be classified into
requirements about integration, (meaningful) representation,
and (fair) comparison, and that (iv) indirect stakeholders who
Copyright c 2015 for this paper by its authors. Copying permitted for private and academic purposes.
are no system users, but are affected by the use of the system
and influence its success, are important.</p>
      <p>This paper is outlined as follows. We first provide
background on the case study and related work on RE with regard
to sustainable development. In Sect. III we describe our research
approach, the study design and the research questions. The
results and threats to validity are presented in Sect. IV, followed
by a discussion in Sect. V and conclusion and future work in
Sect. VI.</p>
    </sec>
    <sec id="sec-2">
      <title>II. BACKGROUND AND EXISTING WORK</title>
    </sec>
    <sec id="sec-3">
      <title>In this section, we provide information about the case study</title>
      <p>as well as relevant background knowledge for readers less
familiar with RE in the context of sustainable development.
Further, we describe some earlier RE research which is relevant
in the context of our study.</p>
      <p>A. The Case Study
emitted less CO2 than possible alternatives. To achieve this
goal, the Compass Group (Switzerland) decided to order an
extension of the existing DSS with game-based techniques
for further processing the calculated LCA data and presenting
them to the meal planners in a recurrent report.</p>
      <p>3) The Context of our Case Study: The case study that
we present in this paper contributes to the second part of
the CarbonFoodPrint project. In the framework of a research
collaboration with Eaternity and the Compass Group
(Switzerland), the sustainability requirements for the new system were
elicited and studied by the first author of this paper, together
with a graduate student. As the focus on the CO2 emission
caused by the production and transportation of ingredients
was given, our study is confined to sustainability requirements
with respect to CO2 which, actually, is only one aspect of
sustainable development.</p>
      <sec id="sec-3-1">
        <title>B. Requirements Engineering and Sustainability Requirements</title>
        <p>
          This case study is part of the project CarbonFoodPrint1 Software systems are embedded in their environment which
initiated by Eaternity2, a company that provides software- by nature is in a permanent process of change. As such,
supported solutions helping people to make their own eating developing and evolving a sustainable software system is an
habits climate-friendly. In this project Eaternity collaborates ongoing process. Typically, the decisions that shape a software
with the Swiss branch of the Compass Group 3, a world-leading system are taken during RE [
          <xref ref-type="bibr" rid="ref3">3</xref>
          ].
food service company. 1) Sustainability: The term sustainability has been used
1) The Current State: The process of planning meals that in different contexts and overused for several purposes. To
is applied within the Compass Group (Switzerland) consists avoid any misunderstanding, we briefly define the terminology
of two phases: First, a small culinary development team in used in this paper. Based on the “Brundtland definition” [
          <xref ref-type="bibr" rid="ref12">12</xref>
          ],
the headquarters composes and develops meal propositions Christen proposes to conceptualize sustainability as an “attempt
for all canteens for the forthcoming months. Based on the to grant the right to a decent life to all living human beings
categories offered in a canteen (e.g. meat, veggie, low carb) without jeopardizing the opportunity to live decently in future”
the proposed meals are accordingly filtered and sent over ( [
          <xref ref-type="bibr" rid="ref13">13</xref>
          ], p. 2). As such he emphasizes that sustainability does not
an SAP-based system to the canteens. In the second phase, solely focus on future generations, but also on human beings
the meal planners working in the canteens adapt the meals living now. He also argues that sustainability is not limited to
based on specific canteen-related criteria, (e.g., their budget and sustaining aspects, but also addresses enabling aspects. By
customers’ preferences). A decision support system (DSS) is definition, sustainability is a global (temporal and spatial)
used to support the meal planners to plan meals by providing concept which makes it clear that no single technology can be
specific functions; in particular, the possibility of selecting sustainable in this sense. However, technology can support the
ingredients (e.g., tomatoes), meal components (e.g., tomato transition towards sustainable development [
          <xref ref-type="bibr" rid="ref4">4</xref>
          ]. Based on this
sauce or spaghetti) and whole meals (e.g., spaghetti with tomato notion of sustainability we define the following terms.
sauce) from a large recipe database, together with corresponding A sustainable software system is a software system that
information on nutrition factors and costs. supports the transition towards sustainable development.
        </p>
        <p>
          2) The Project Context: The overall goal of the CarbonFood- A sustainability requirement is a requirement for a
susPrint project is to motivate meal planners to select ingredients tainable software system which concerns sustainability.
whose production and transportation emitted less CO2 than A positive enabling effect is an effect that is enabled by
possible alternatives. The project consists of two parts. The first using a software system and positively contributes to the
part focuses on the calculation of CO2 emissions by applying transition towards sustainable development.
life-cycle assessment (LCA) [
          <xref ref-type="bibr" rid="ref11">11</xref>
          ]. This is a specific technique to Different metaphorical descriptions of the roles of
environaddress the environmental aspects and potential environmental ment, society, and economy in sustainable development exist.
impacts such as use of resources and the environmental As the economic system is part of human society, which in
consequences of releases throughout a product’s life cycle. turn is part of the environment, we use the metaphor of nested
The second part of the project focuses on the utilization and circles, where economy is represented by the inner circle,
representation of these data to motivate the meal planners to society by the middle one and environment by the outer one.
select ingredients whose production and transportation have For more information see for example Hilty &amp; Aebischer [
          <xref ref-type="bibr" rid="ref4">4</xref>
          ].
1http://bit.ly/1L2Qzcc 2) State of Research: The high importance of addressing
2http://eaternity.com the concept of sustainable development within the process
3http://welcome.compass-group.ch/ of requirements elicitation has been recognized by the RE
community, e.g., [
          <xref ref-type="bibr" rid="ref6">6</xref>
          ] [
          <xref ref-type="bibr" rid="ref8">8</xref>
          ] [
          <xref ref-type="bibr" rid="ref9">9</xref>
          ]. However, as noted by Becker [
          <xref ref-type="bibr" rid="ref14">14</xref>
          ], in the context of a project for extending an existing software
this has not yet been transferred into practice successfully. system with game-based mechanics.
        </p>
        <p>Prior RE research has mainly conceptualized sustainability Research Questions. From this goal we derived two
rerequirements as quality requirements (i.e., a sub-category of search questions:
specific quality requirements in the taxonomy introduced by RQ1: What is specific about requirements concerning positive</p>
      </sec>
      <sec id="sec-3-2">
        <title>Glinz [15]). enabling effects?</title>
        <p>
          Thereby, the focus was on goal modeling processes e.g. RQ2: How can game-based mechanics motivate positive
by regarding sustainability as a trade-off between business enabling effects when extending existing software systems?
goals [
          <xref ref-type="bibr" rid="ref16">16</xref>
          ], by using the idea of generic goal refinement as a
checklist for sustainability requirements [
          <xref ref-type="bibr" rid="ref3">3</xref>
          ], by treating them B. Study Design
similarly to conflicting goals of budget restrictions and quality
improvements [
          <xref ref-type="bibr" rid="ref17">17</xref>
          ], by suggestion how to align the objective of
environmental sustainability with the other objectives [
          <xref ref-type="bibr" rid="ref1">1</xref>
          ], and
by building upon different levels of impacts [
          <xref ref-type="bibr" rid="ref18">18</xref>
          ]. Further,
Roher &amp; Richardson work on patterns for sustainability
requirements [
          <xref ref-type="bibr" rid="ref19">19</xref>
          ].
        </p>
        <p>
          We followed a “mixed methods” approach [
          <xref ref-type="bibr" rid="ref23">23</xref>
          ], consisting
of three sequential steps: a contextual inquiry, semi-structured
interviews and an online questionnaire. We used this approach
for investigating our research questions from more than one
perspective, thus getting more thorough results. As mentioned
in Sect. II.A, the tasks of eliciting the requirements and
conducting the study were both performed by the first author
C. Sustainability Requirements Related to Enabling Effects of this paper, together with a graduate student.
        </p>
        <p>
          Most existing work on sustainability requirmements as 1) Study Setup: The process for all three steps and the
described above focus on direct effects (cf. the classification questions to be asked were elaborated by the first author of
presented in the introduction) and treat sustainability require- this paper with support from the graduate student, then reviewed
ments as goals or as specific quality requirements. Sustainability by a group of RE researchers and finally improved according
requirements related to enabling effects of a software system to the feedback received. The interviews and the questionnaire
are far less addressed both in research and – to our experience were both piloted with people neither specifically related to the
– also in industry. A possible explanation is that in most cases domain of RE nor to the one of sustainability. This approach
direct effects are directly connected to economic goals, whereas was chosen to make sure the questions are clear to participants
for enabling effects, such a relation is hard to establish in most who are unfamiliar with these domains [
          <xref ref-type="bibr" rid="ref24">24</xref>
          ].
cases. The study was carried out over a period of four months and
        </p>
        <p>
          Nevertheless, as outlined by Wang [
          <xref ref-type="bibr" rid="ref20">20</xref>
          ], research shows included eight steps: (1) Preparing the contextual inquiry, (2)
that while considerations about sustainable development are carrying out the contextual inquiry and evaluating the results,
becoming more relevant in societies, positive environmental (3) designing the interview questions, (4) conducting pilot
effects positively impact the value chain and the image of interviews, (5) carrying out the interviews and evaluating the
a company. Consequently, requirements related to enabling results, (6) designing the questions following the guidelines for
effects of software systems increasingly gain relevance for creating a questionnaire by Kitchenham &amp; Pfleeger [
          <xref ref-type="bibr" rid="ref24">24</xref>
          ], (7)
both companies and their stakeholders. publishing the questionnaire, (8) evaluating the questionnaire
results. Note that the results of the pilot interviews were not
III. RESEARCH METHODOLOGY AND STUDY DESIGN included in the data analysis.
        </p>
        <p>
          For choosing our research methodology, we considered the All steps of the study were conducted in German.
Consefollowing facts: (i) We wanted to analyze and better understand quently, the questions and interviewee quotes reported in this
sustainability requirements with respect to positive enabling paper are our translations of the German originals.
effects, (ii) there is little knowledge available about this kind 2) Selection of Participants: The Compass Group
(Switzerof requirements, (iii) we had the opportunity to study a real land) selected the participants for the contextual inquiry and
industrial project in this context. Given this situation, we chose the interviews. However, we could provide our criteria for the
an exploratory case study as our research methodology. Such a selection process. The following of our criteria were accepted
study enables an in-depth investigation of a phenomenon in its and applied by the company: (1) all participants are responsible
context [
          <xref ref-type="bibr" rid="ref21">21</xref>
          ] and is specifically suitable when little knowledge for the meal planning process and as such are direct end-users
about the subject is available [
          <xref ref-type="bibr" rid="ref22">22</xref>
          ]. Further, the results of an of the DSS; (2) the contextual inquiriy is conducted in two
exploratory study form the basis for both theory generation [
          <xref ref-type="bibr" rid="ref21">21</xref>
          ] sessions, one in the headquarters of the company and one in
and constructive solution design. a canteen; (3) at least 15 meal planners are selected for the
interviews; (4) the group of participants is heterogeneous with
A. Research Goal and Research Questions regard to canteen size, region, and work sector (“Business &amp;
        </p>
        <p>According to our research plan and the given project context, Industry” (B&amp;I) and “Eductation” (Edu)). We did not include
we formulated our research goal as follows. gender, age or nationality into our selection criteria, since we</p>
        <p>Goal. Analyze sustainability requirements for the purpose considered them as irrelevant for the purpose of the study.
of developing sustainable software systems with respect to Eventually, 19 meal planners working in different canteens
positive enabling effects from the viewpoint of the end-users participated in the semi-structured interviews.</p>
        <p>The URL of the online questionnaire was sent to all student and then discussed between them in a retrospection
meal planners (about 150). 67 participants finished the on- session. As mentioned above, the interview questions as well
line questionnaire. However, seven of them did not answer as the interviews were piloted before conducting the actual
the question about current reasons for changing ingredients. interviews.</p>
        <p>
          The questionnaire results presented here come from the 60 5) Online Questionnaire: We used the results from the
meal planners who answered all questions (this includes all interviews as a basis for designing the online questionnaire.
participants of the interviews and the contextual inquiry). Our goal was to elicit quantitative information about important
3) Contextual Inquiry: To understand the current situation, aspects of sustainability requirements from a sample of involved
i.e., how meal planners currently work and apply the existing people which is larger than the number of meal planners
DSS, we conducted a contextual inquiry in two sessions: interviewed. We exploited the majority of the interview
the first one at the headquarters of the Compass Group questions in the online questionnaire, omitting the ones that
(Switzerland) and the second one in one of the canteens. focus on the end-user’s attitude towards the project. Table I
Contextual inquiry [
          <xref ref-type="bibr" rid="ref25">25</xref>
          ] is an elicitation technique that studies shows four representative questions (QQ-1–QQ-4) that we
stakeholders in the field, bringing the requirements engineer in further analyze in this study. Semantic differential scales [
          <xref ref-type="bibr" rid="ref29">29</xref>
          ]
contact with the stakeholders in their real work environment. were applied to evaluate the participants’ attitude. This type
Thereby, the requirements engineer takes a role similar to of scale is similar to the Likert scale [
          <xref ref-type="bibr" rid="ref30">30</xref>
          ] with the benefit of
the one of an apprentice, asking questions while observing revealing both the direction and the intensity of each opinion.
the work process. By allowing requirements engineers and For questions about familiar topics we used an even scale
stakeholder representatives to work together and to share (four point), for questions where we expected less or
noninsights, a contextual inquiry enables a full understanding familiarity, we used an odd scale (three or five point, including a
of the work practices in the specific work environment. As neutral point). This approach is generally suggested for defining
a contextual inquiry is exploratory and open-ended, we did the number of alternatives given in ordinal scales [
          <xref ref-type="bibr" rid="ref31">31</xref>
          ]. The
not prepare questions beforehand, with the exception of some link to the questionnaire was published over the intranet of
ice-breaking questions for starting the inquiry sessions. the Compass Group (Switzerland) together with background
4) Semi-Structured Interviews: The questions for the inter- information about the project, who we are, the goal of the
views were grouped into three parts: demographics and current study and criteria for participation.
work routines, usability, and motivation. While the questions Technically, we used an online questionnaire tool5 for
creatabout the first two parts build on the results of the contextual ing the questionnaire. The full set of questionnaire questions
inquiry, the ones about motivation regarding the game-based (in German) and the questionnaire design are available at 6.
aspects refer to the results from our previous research on C. Collecting the Data
requirements for game-based approaches motivating sustainable
consumption [
          <xref ref-type="bibr" rid="ref26">26</xref>
          ]. Four representative interview questions are Data collection started in July 2014 with the two contextual
given in Table I. IQ-1 is from part one about current work inquiry sessions and ended in October 2014 when the online
routines, IQ-2 is from part two about usability, and IQ-3 as questionnaire was online for two weeks.
well as I-Q4 are from part three about motivation, however, The full data set has a size of 81, comprising the data from
I-Q4 also affects part two. The full set of interview questions two contextual inquiry sessions, 19 interviews and 60 fully
(in German) is available at 4. completed questionnaires. It turned out that the first contextual
        </p>
        <p>
          The interviews lasted between 25 and 45 minutes on average. inquiry session had primarily served for making the researchers
The time difference can be explained by our approach of familiar with the context of the project, so we excluded it from
conducting semi-structured interviews with a mix of open- the data set. Further, one interview was not recorded due to a
ended and specific questions. We did this for enabling the technical problem which we realized only after the interview
elicitation of both foreseen and unexpected information [
          <xref ref-type="bibr" rid="ref27">27</xref>
          ]. was finished. Hence, we also excluded that interview from
This was specifically relevant since the domain knowledge the data set. All interviewees as well as the participant of the
of favorabale enabling effects is still immature in RE. It also contextual inquiry also answered the questions in the online
helped to build a positive rapport with the interviewees [
          <xref ref-type="bibr" rid="ref28">28</xref>
          ]. questionnaire. So we have a total of 79 data points from a total
        </p>
        <p>All interviews were conducted over Skype by calling the of 60 participants for analysis. With respect to the canteen
interviewees on their business phones. This approach was sectors, i.e., Business &amp; Industry (B&amp;I) vs. Education (Edu),
chosen because the interviewees were distributed over the the data are distributed as follows: (i) contextual inquiry session:
whole country. So visiting them all would have been too costly zero B&amp;I, one Edu; (ii) interviews: fourteen B&amp;I, four Edu;
and also not possible in the timeframe given for the interviews. (iii) online questionnaire: fifty B&amp;I, ten Edu.
Half of all interviews were conducted by the first author of D. Participant Demographics
this paper, the other half by her graduate student. To align Some demographic information about the participants is
the interview styles and reduce observer bias, the first two summarized in Table II. 83% of all participants work in
interviews were conducted jointly by the first author and her</p>
      </sec>
    </sec>
    <sec id="sec-4">
      <title>4http://bit.ly/1QyuR3i</title>
    </sec>
    <sec id="sec-5">
      <title>5http://ww3.unipark.de 6http://bit.ly/1FiFKdO</title>
      <p>Identifier Original</p>
      <p>Question Id1
IQ-1 2.d
IQ-2 3.a
IQ-3
IQ-4
QQ-1
QQ-2
QQ-3
QQ-4
4.a
6
4.1.1
5.3
7.1.1
6.1</p>
      <p>Question (English translation)
According to what criteria do you compose a meal?
Can you imagine that the idea of reducing the CO2 value of meals due to the choice of ingredients is realizable in your
canteen?
Are you additionally to the CO2 value interested in the following information? i.What the CO2 value means e.g. how many
kilometers driven by car do correspond to it? i.1 Are you further interested in other representations? If so in which?
Do you believe that the CO2 values of different canteens can be compared with each other in such a report?
What does motivate you to change a meal? Formulated in a clearer way, how strong does one of your changes affecting a
meal component or ingredient depend on the following criteria? (costs, variety, customer preference, season, compliance,
environmental aspects)
How should the CO2 emission value of your meals additionally be represented in order to raise your interest? (number of
kilometers a mean of transportation has to make; amount of energy an ordinary private household has to use; number of days
an ordinary private household has to be heated in order to emit the same amount of CO2; number of trees that are needed to
compensate the same amount of CO2);
How good do you think is the following information? Segmentation of the CO2 based on the process steps; meal components;
origin of ingredients
How strong have the following factors to be considered in order you perceive the comparison of the CO2 emission of different
canteens to be fair? (size of the canteen according to the number of cooked meals; the number of employees working in the
kitchen; the number of meal categories served in a canteen; the customer preferences)
“IQ” refers to the interview questions and “QQ” to the questionnaire questions. The column “Original Question Id” refers to the corresponding original question
id of the interview or the questionnaire, available under 1: http://www.ifi.uzh.ch/rerg/research/stakeholderengagement/garuso
B&amp;I canteens and 17% in Edu canteens. On average, the
participants have more than five years experience in their
position. Regarding meal variety and number, participants
working in B&amp;I canteens have three meal categories on
average (e.g., meal with meat, vegetarian, low-carb), while
participants working in Edu canteens have four. Around 40%
of all participants from both sectors work in canteens that
produce between 150 and 499 meals per day. 42% of the B&amp;I
canteens and 20% of the Edu canteens produce less than 150
meals per day, and more than 500 meals are daily produced in
16% of the B&amp;I canteens and in 40% of the Edu canteens.</p>
      <sec id="sec-5-1">
        <title>E. Analyzing the Data</title>
        <p>
          All data is exploratory, which means we did not pre-specify
a hypothesis as it is done in a confirmatory analysis. The
end-product of exploratory data analysis is rather suggesting
patterns for further studies and providing hypothetical insight
into these patterns instead of statistical figures [
          <xref ref-type="bibr" rid="ref32">32</xref>
          ]. As such
we did not apply any statistical tests, but provide first insights
into sustainability requirements for positive enabling effects
with regard to sustainable development.
        </p>
        <p>1) Contextual Inquiry: Information from the contextual
inquiry sessions was first structured to identify main work
processes which then were used for defining the interview
questions. In the presentation of the results in the next section,
the data from the contextual inquiry is not analyzed separately,
but presented together with the data from the interviews.</p>
        <p>
          2) Semi-Structured Interviews: To be able to better evaluate
the data from the interviews we structured the data by first
transcribing and then coding them according to the process
described by Seaman [
          <xref ref-type="bibr" rid="ref33">33</xref>
          ]. The corresponding codes are listed
in Table III. The results of this analysis were used to define
the questions and structure of the online questionnaire.
        </p>
        <p>3) Online Questionnaire: As it is common in exploratory
studies, we visually analyzed the quantitative data from the
questionnaires, choosing divergent stacked bar charts.</p>
      </sec>
    </sec>
    <sec id="sec-6">
      <title>IV. RESULTS</title>
    </sec>
    <sec id="sec-7">
      <title>In this section we present our results. As usual when</title>
      <p>presenting qualitative results, the data are complemented with
quotations from the interviewees. The quotations are written
in italic, the interviewee who stated the quote is indicated
in brackets by “I-” followed by the number of the interview.
We coded relevant information in the qualitative results to
structure and quantify them; these codes are underlined and in
brackets. Table III lists the sixteen codes that we used, together
with the frequency of their appearance in the interviews. The
questionnaire results for QQ-1–QQ-4 (cf. Table I) are visualized
in Figures 1-4 in the same order as the corresponding questions.</p>
      <p>We present the results grouped by topics. Within each topic,
we first present the results from the interviews regarding this
topic, and then the corresponding results from the online
questionnaire.</p>
      <sec id="sec-7-1">
        <title>A. Requirements Looked through the Sustainability Lens</title>
        <p>The first aspect we investigated was whether and how the
current set of requirements changes from the end-users’
perspective when adding the dimension of sustainable development to
the domain context (RQ1). We first focused on actually existing
requirements causing the participants to change ingredients in
their current work process. Second, we elicited requirements
that the participants perceive as important if they had to select
ingredients with respect to CO2 emission. This subsection
highlights the aggregated results related to this aspect.</p>
      </sec>
      <sec id="sec-7-2">
        <title>1) Current Requirements Motivating Change: We specif</title>
        <p>ically asked the interviewees about reasons for changing
ingredients in the proposed meals they get from the culinary
development team (IQ-1). The three criteria mostly mentioned
are customers’ preferences, costs and variety.</p>
        <p>As indicated in Table III for thirteen interviewees
customers’ preferences (changeIngredients_customers_current)
are a strong reason to change ingredients, twelve
emphasized the relevance of cost restrictions
(changeIngredients_costs_current) and six mentioned the
variety (changeIngredients_variety_current) of their meals as
an important reason. Interviewee I-8, for example, highlighted
both variety and customers’ preferences: “I do it according to
the following criteria, such that there is variety. Theoretically,
pork is the cheapest meat we can get, but I nevertheless look
that it is only served once a week. Also a little bit because of
our Muslims (...)”.</p>
        <p>In the online questionnaire, we further explored these results
by asking the participants to rate the importance of costs,
variety, customers’ preferences, together with the seasons,
compliance with suppliers, and the environment as a reason
for them to change the ingredients (QQ-1). Participants could
rate the importance on a semantic differential scale of four
criteria: “Plays no role at all”, “Plays a minor role”, “Certainly
plays a role”, “This criterion is one of the most important
reasons for change”. The results shown in Fig. 1 support what
we found in the interviews: Customers’ preferences and costs
are the two most important reasons to change ingredients in
both Edu and B&amp;I canteens. Variety and season come next.
Participants working in Edu canteens rate these two criteria as
equally important, while participants working in B&amp;I canteens
rate season to be slightly more important than variety.</p>
        <p>2) Considering Sustainability Requirements: When asked
the interviewees whether they can imagine reducing the CO2
emission of their meals by selecting specific ingredients
(IQ2), thirteen were positive about having the potential to do
so (successCO2Red_changingIngredients_new). However, the
interviewees agreed that this goal is only achievable when
considering certain constraints. The results described below
show that (i) costs and customers’ preferences are perceived as
the two most relevant constraints if the context of sustainable
development is added; (ii) the influence of these two constraints
becomes stronger in this context and new constraints become
relevant; (iii) an integration of the game-based extension into
the existing DSS is important.</p>
      </sec>
    </sec>
    <sec id="sec-8">
      <title>Cost constraints (constraintCO2Red_costs_new) were stated</title>
      <p>by nine interviewees to challenge the successful consideration
of CO2 emission when selecting ingredients. Three of them
regard cost restrictions as a major criteria for a project failure.
The rest of them is not that strict, however they mentioned
that local and organic products usually are more expensive
than non-organic products or products from farther away. They
highlighted that if their budget remains the same, customers
had to pay the cost difference. Interviewee I-17 considered the
relevance of a company-wide change: “Well it [the success]
depends on what kind of food it is. In my opinion, if costs
raise, we have to pass these costs on to our customers in order
to stay in the green zone [with the costs]. (...) Actually, the
whole company should have to participate.”. I-14 stated: “(...)</p>
      <sec id="sec-8-1">
        <title>We have to get the cheapest products in order to fulfill the terms of the company and meet the demands of the customers (...) the customers have to rethink because, if I buy a regional product, this has immediate influence on my costs. ”.</title>
        <p>Customers’ preferences are further regarded as a
constraint that challenges the success of the project
(constraintCO2Red_customers_new) by nine interviewees.
Seven of them can imagine to possibly manage this challenge
by involving the customers (e.g., by explaining the effects). The
other two do not think that customers will change their eating
habits or pay more for climate-friendly food. I-14 stated for
example: “Well, the cooks would like to consider this [the CO2
emission related to the ingredients]. It has been a nonsense to
offer tomatoes in December. However, it is a fact that not us,
but the customers do rule the market (...) In my opinion the
% of respondents
100 80 60 40 20 0 20 40 60 80 100</p>
        <p>Plays no role at all Plays a minor role Certainly plays a role
This criterium is one of the most important reasons for change
sn #trees for compensation
du tee #heating days in a household
-E an#kilometers by any mean of transportation</p>
        <p>C Eletricity used in a private household</p>
        <p>#trees for compensation
issseun tseeann#kilometers by any mean of transportation</p>
        <p>#heating days in a household
-B C Eletricity used in a private household</p>
        <p>This makes absolutely no sense
I regard this as rather meaningful
% of respondents
100 80 60 40 20 0 20 40 60 80 100
This seams not very meaningful I cannot answer this question
Absolutely, this is absolutely meaningful
awareness of the customers is not yet there.”</p>
        <p>The size of the work space was highlighted by two
interviewees (constraintCO2Red_space_new) as a major constraint.
Little space in the kitchen affects work processes and as such
the choice of ingredients. I-12 put it this way: “Our restaurants
are not all equipped in the same way (...) Some [of us] really
must additionally plan their meals based on the available space
[to process the food] and the storage possibilities and then
decide whether they buy fresh broccoli or frozen broccoli.”</p>
        <p>The proper integration
(successCO2Red_integratedInWorkProcess_new) of the game-based extension in the
existing DSS is regarded as highly important by thirteen
interviewees. For example, six of them said that it is highly
important to have immediate access to alternative ingredients
with a lower CO2 emission value while planning meals over
the system (successCO2Red_alternativesShown_new). I-2
emphasized: “(...) there really must be alternatives with which
the CO2 emission value can be reduced. Showing a direct
alternative in the sense of ‘tomatoes from Italy instead of
cherry tomatoes from overseas’ should be possible.” In this
context, I-7 highlighted time pressure as a reason: "When
we get the [suggested] meal plan, it [the system] has to be
ready with the CO2 emission values. Because we cannot
search extensively for this information on the lists of the
vegetable, meat, or fish suppliers (...) this [information] has to
be integrated and then we can work.”</p>
      </sec>
      <sec id="sec-8-2">
        <title>B. Game-based Mechanics for Positive Enabling Effects</title>
        <p>The second aspect we investigated was about how
gamebased mechanics have to be applied to motivate positive
enabling effects in the given context (RQ2). Our focus was on
a meaningful representation of the CO2 emission value and on
relevant factors for comparing the values of different canteens
with each other. In this sub-section we present the aggregated
results.</p>
        <p>We asked the interviewees if the CO2 emission shall be
represented (successCO2Red_CO2Rep_new) by more familiar
measures (IQ-3) and provided the number of kilometers driven
(i.e., how many kilometers one could drive for emitting the same
amount of CO2) as an example. Seventeen interviewees agreed
on the importance of a meaningful representation. I-12 added
that such a value would even be more meaningful if represented
by a journey. “It would be good if you could say, we have
saved that amount of CO2, this is enough to travel from here to</p>
      </sec>
      <sec id="sec-8-3">
        <title>Moscow or simply, you could travel that far with this amount</title>
        <p>(...) because we do not know what consumes how much [CO2].</p>
      </sec>
      <sec id="sec-8-4">
        <title>For example, when I drive home with my car, I know I produce</title>
      </sec>
      <sec id="sec-8-5">
        <title>CO2, but I do not know how much. And this is the reason why</title>
        <p>we don’t know what is a good value and what is a bad one.”
Five interviewees highlighted the relevance of using graphics
and pictures (successCO2Red_CO2RepGraphically_new). I-14
explained it this way, “(...) something like ’[with this amount
of CO2] you could have driven from here to there with a truck’
and then showing something similar to Google maps. This
way, it becomes visible right away that I could have driven to</p>
      </sec>
      <sec id="sec-8-6">
        <title>Marseilles.”</title>
        <p>The questionnaire results underline these results. Together
with the given example on the amount of kilometers we
provided three more representation options to rate for (QQ-2):
the number of trees needed to compensate the amount of CO2
emitted, the used heating energy, and the used electricity in
an average household (Fig. 2). These four options could be
rated on a semantic differential scale of 5 criteria: “This makes
absolutely no sense”, “This seams not very meaningful”, “I
can’t answer this question”, “I regard this as rather meaningful”,
and “Absolutely, I think this is absolutely meaningful”. The
results show that the number of trees needed for compensation
and the number of kilometers that can be driven are similarly
perceived as highly meaningful for representing the amount of
CO2 emissions. Participants from both sectors were much less
interested in a representation by the energy consumption in a
household of both electricity and heating.</p>
        <p>Segmentation of the CO2 value was raised by four
interviewees. Two of them wanted a segmentation based on the origin
of ingredients (successCO2Red_CO2PerOrigin_new) e.g., the
CO2 value of a tomato from Greece compared to one from Italy.
Three interviewees were interested in segmenting the CO2 value
according to the two process steps of production and
transportation (successCO2Red_CO2PerProcessSteps_new), and all
four found it relevant to segment the meals with regard to
their components (successCO2Red_CO2PerComponents_new),
e.g., the two components pasta and pasta sauce. I-14 made
this very clear: “When you do something like this [the project
as a whole], I think it is good that we have a learning effect,
something like ’Ahaaaaaa, there it is, this shrimp has messed
up our whole meal statistics (...) This way rethinking happens.</p>
      </sec>
      <sec id="sec-8-7">
        <title>We are a bit practice-oriented.”</title>
        <p>These results were exploited further in the questionnaire by
asking to rate the segmentation factors origin, meal components,
s
n
du tee
and process steps (QQ-3) on a semantic differential scale of
four criteria: “I don’t like this at all”, “I think this is rather
bad”, “I think this is rather good”, “Absolutely, I think this is
awesome”. The results show that the majority of both sectors
is interested in comparing ingredients based on their origin
(Fig. 3).</p>
        <p>As game-based techniques typically involve comparisons, we
specifically asked whether the CO2 value of different canteens
can be compared with each other (IQ-4); thirteen interviewees
said yes (comparisonCO2Possible_gamification_new). Eight of
them stated that the most important condition to consider for
enabling comparisons between canteens is the number of meals
produced (comparisonCO2Possible_numberOfMeals_new).</p>
      </sec>
      <sec id="sec-8-8">
        <title>I-17 got very specific: “You cannot compare a small canteen</title>
        <p>with a large one (...) If I compare two small canteens with
each other that sell on average approximately sixty to eighty
meals per day, then I think this is comparable. (...) I think here
we should differentiate such that we only compare very similar
canteens, also with respect to the location.”</p>
        <p>These interview results are supported by the results from
the questionnaire (Fig. 4). The participants could rate the
importance of number of meals, of employees working in the
kitchen, of meal categories offered, location of the canteen,
customers’ preferences for comparison (QQ-4), on a semantic
differential scale of four with the two opposite criteria: “Plays
no role at all” and “Plays a significant role”. The majority of
participants from B&amp;I canteens selected the number of meals,
followed by customers’ preferences to enable fair comparison,
while the majority of participants form Edu canteens selected
customers’ preference as the top criterion, followed by the
number of meals.</p>
      </sec>
      <sec id="sec-8-9">
        <title>C. Threats to Validity</title>
        <p>
          We discuss the threats to validity using the usual four
categories: internal, external, construct, and conclusion validity
[
          <xref ref-type="bibr" rid="ref34">34</xref>
          ].
        </p>
        <p>Internal Validity reflects the relationship between cause and
effect. The internal validity of our study is limited due to its very
nature: in an exploratory case study in a real world project,
many potentially confounding factors cannot be controlled.
A potential threat is the fact that the persons participating
in the contextual inquiry as well as the interviewees were
selected by the Compass Group (Switzerland). We do not
consider this a major threat because we provided the company
with our selection criteria and also did not find any evidence
for selection bias when analyzing the interviews. Answering
the online questionnaire was voluntary, which may cause the
results to be biased due to highly supportive or highly
nonsupportive participants. Since the subject of the study was
equally interesting for both supporters and opponents, we do
not believe that the outcome of our study is affected by this
potential threat. We mitigated potential maturation problems by
scheduling the interviews during working hours and limiting
their duration to 20-40 minutes.</p>
        <p>External Validity refers to the extent of being able to
generalize the results. The biggest limitation to the external
validity of our study is the fact that (i) the study was conducted
in a single company and (ii) the project was limited to positive
enabling effects with respect to CO2 emission. However, the
setting of our study is typical for systems in the service domain:
employees of a service company use a software-based system as
end-users to provide services to the customers of the company,
while the customers’ preferences as well as cost considerations
have a major influence on the system’s requirements. Further,
reducing CO2 emission is a problem that, in our opinion, can
be considered to be representative for the whole domain of
problems considering sustainable development. Based on these
two reasons, we argue that our results are generalizable at least
to some extent to positive enabling effects about sustainability
in general for systems in the service domain.</p>
        <p>Construct Validity describes our ability to measure what
we actually intend to measure. All participants had the
same tasks. However, we believe to have reduced
monooperation bias by including different work locations and work
domains (Business &amp; Industry, Education) into our study. We
have minimized mono-method bias by using methodological
triangulation (contextual inquiry, interview, questionnaire). To
avoid evaluation stress, we assured all participants that their
data were treated confidentially and evaluated for research
purposes only.</p>
        <p>Conclusion Validity is concerned with drawing correct
conclusions based on our observations. The first author was
involved in designing the study and executing the elicitation
techniques, which could potentially cause observer bias. For
mitigating this threat, we used methodological and observer
triangulation and reviewed the structure and questions of all
three elicitation methods with a group of experienced RE
researchers. By conducting several pilot studies we strengthened
the quality of our wording. Further, we encouraged the
interviewees to ask for clarification if something was unclear.
Therefore, we do not consider measure reliability as a major
threat.</p>
      </sec>
    </sec>
    <sec id="sec-9">
      <title>V. DISCUSSION</title>
    </sec>
    <sec id="sec-10">
      <title>In this section, we discuss the results of our study with respect to our two research questions and present some key findings.</title>
      <sec id="sec-10-1">
        <title>A. RQ1: What is specific about requirements concerning positive enabling effects?</title>
        <p>The results of our study show that the requirements do
change when sustainability comes into play. In our study, when
extending the existing DSS for meal planning with
gamebased mechanics for motivating environment-friendly choices
to achieve a reduction of CO2 emissions, we mainly found three
kinds of changes in requirements: new constraints, existing
constraints that become more important, and new functional
requirements.</p>
        <p>For example, the size of the kitchen is a new constraint that
the interviewees only perceive when the system is used in the
context of positive enabling effects for achieving a reduction
of CO2 emission. The smaller the kitchen, the less fresh food
can be processed, which means that the meal planners have to
go for more frozen or pre-processed food. This, in turn, has
an effect on the CO2 footprint of the meals.</p>
        <p>Participants from both Edu and B&amp;I canteens identified
customer preferences and cost restrictions as the most important
constraints. The interviewees emphasized their fear about
increased cost constraints in the context of positive enabling
effects due to higher prices for local and organic food. Further,
they are afraid of loosing customers by excluding off-season
products, which results in a smaller variety of food. That means
that the existing constraints of customer preferences and cost
restrictions become more important in a sustainability context.</p>
        <p>72 percent of all interviewees emphasize the need to integrate
the information about CO2 emission values into the DSS. 33
percent specifically want to be able to immediately access a list
of alternative ingredients with respect to their CO2 footprint.
These are new functional requirements.</p>
        <p>Consequently, treating sustainability requirements as a
subcategory of specific quality requirements (cf. Section II-B2)
turns out to be inadequate. Requirements concerning positive
enabling effects can be functional requirements or constraints.</p>
        <p>The growing importance of customer issues in the context
of positive enabling effects such as customer preferences and
cost restrictions also provides evidence that in the context of
sustainability requirements, there is a strong need for taking
into account also the indirect stakeholders of a system (i.e.,
those who are not end-users of the system, but are affected by
its use, e.g., customers).</p>
      </sec>
      <sec id="sec-10-2">
        <title>B. RQ2: How can game-based mechanics motivate positive enabling effects when extending existing software systems?</title>
      </sec>
    </sec>
    <sec id="sec-11">
      <title>The results show that it is important to integrate the game</title>
      <p>based mechanics directly into the underlying system and that
end-users have to perceive the representations of sustainability
goals to be meaningful as well as comparisons to be fair.</p>
      <p>As mentioned above, the interviewees emphasize the need
to integrate the information about CO2 emission values into
the DSS and want to be able to immediately access a list of
alternative ingredients with respect to their CO2 footprint.</p>
      <p>The requirement of representing the CO2 values in a
meaningful way is perceived as relevant by 94% of all interviewees.
Metaphorical representations such as number of trees needed
to compensate the CO2 emission or number of kilometers made
by a mean of transportation until the same amount of CO2 is
emitted are preferred over more abstract representations such as
energy equivalents. Interviewees also mentioned the importance
of concrete and visual representations e.g a concrete example
for a trip from city A to city B, preferably indicated on a map.
A proper segmentation of the displayed information is also
important. In our study, most participants favored the origin
of ingredients as segmentation criterion.</p>
      <p>
        As discussed in our previous research [
        <xref ref-type="bibr" rid="ref26">26</xref>
        ], comparison
of individual achievements is an important game mechanic
for motivating sustainable actions. The results of our study
underline the importance of considering the work context to
enable fair comparison when comparing the CO2 footprint of
different canteens with each other. The relevance of customer
preferences as a factor for enabling fair comparison again
underlines the need for taking the needs of indirect stakeholders
into account.
      </p>
      <p>When analyzing the interview data with respect to the codes
that originate from information provided spontaneously by the
interviewees (see Table III), we found that this information (and
hence, the corresponding sustainability requirements) can be
grouped into three categories: (a) integration of sustainability
information into the current system and work process ( i.e., by
properly extending the current software system instead of just
adding a new, separate module), (b) meaningful representation
of the addressed sustainability aspect (CO2 emission in our
study), and (c) fair comparison of the achievements of the
addressed users (meal planners in different canteens in our
study). Although more research is necessary to establish
the generalizability of this finding, our study provides some
evidence that a classification of sustainability requirements with
respect to positive enabling effects into the classes integration,
(meaningful) representation and (fair) comparison makes sense.</p>
      <sec id="sec-11-1">
        <title>C. Key Findings</title>
      </sec>
    </sec>
    <sec id="sec-12">
      <title>In summary, we draw five key findings about sustainability requirements regarding favorable enabling effects from the results of our study. In the context of positive enabling effects regarding sustainable development:</title>
      <p>Requirements for a software system do change when
sustainable development is considered.</p>
      <p>We find both new requirements and existing requirements
that become more important.</p>
      <p>Considering sustainability requirements to be a subset
of quality requirements is inadequate. We also found
functional requirements and constraints.</p>
    </sec>
    <sec id="sec-13">
      <title>Game-based mechanics need to be integrated directly into the underlying system. Meaningful representations of the sustainability aspect as well as fair comparison are important.</title>
    </sec>
    <sec id="sec-14">
      <title>Further we have two findings where our study provides some</title>
      <p>evidence, but further research is necessary.</p>
    </sec>
    <sec id="sec-15">
      <title>Sustainability requirements can be classified into three</title>
      <p>classes: integration, (meaningful) representation and (fair)
comparison.</p>
      <p>Indirect stakeholders, i.e., those affected by the use of
the deployed system, should be involved when eliciting
sustainability requirements.</p>
    </sec>
    <sec id="sec-16">
      <title>VI. CONCLUSION AND FUTURE WORK</title>
      <p>We reported on the results of elicited sustainability
requirements regarding positive enabling effects. The study includes
78 data points from 60 participants working in 60 different
canteens. Our main contribution is to reveal differences of
such sustainability requirements compared to requirements
in traditional settings, as well as important requirements to
consider in a context of sustainability requirements for favorable
enabling effects. Moreover, we found evidence that indirect
stakeholders are important in this context and a possible
classification of sustainability requirements.</p>
      <p>In our future work we will further exploit the findings of
this study. In particular, we plan to investigate the elicitation
of sustainability requirements from indirect stakeholders who
are outside of organizational reach.</p>
    </sec>
    <sec id="sec-17">
      <title>ACKNOWLEDGMENT</title>
    </sec>
    <sec id="sec-18">
      <title>The authors thank Eaternity for the opportunity to con</title>
      <p>duct this study and all employees of the Compass Group
(Switzerland) who participated in the study. This research was
partially funded by Gebert Rüf Stiftung under grant number
GRS-023/13.</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          [1]
          <string-name>
            <given-names>B.</given-names>
            <surname>Penzenstadler</surname>
          </string-name>
          , “
          <article-title>Infusing green: Requirements engineering for green in and through software systems</article-title>
          ,
          <source>” in 3rd International Workshop on RE4SuSy</source>
          , vol.
          <volume>1216</volume>
          .
          <string-name>
            <surname>CEUR-WS</surname>
          </string-name>
          ,
          <year>2014</year>
          , pp.
          <fpage>44</fpage>
          -
          <lpage>53</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          [2]
          <string-name>
            <given-names>M.</given-names>
            <surname>Mahaux</surname>
          </string-name>
          and
          <string-name>
            <given-names>C.</given-names>
            <surname>Canon</surname>
          </string-name>
          , “
          <article-title>Integrating the complexity of sustainability in requirements engineering</article-title>
          ,” in
          <source>First International Workshop on Requirements for Sustainable Systems</source>
          ,
          <year>2012</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          [3]
          <string-name>
            <given-names>M.</given-names>
            <surname>Mahaux</surname>
          </string-name>
          ,
          <string-name>
            <given-names>P.</given-names>
            <surname>Heymans</surname>
          </string-name>
          , and G. Saval, “
          <article-title>Discovering sustainability requirements: An experience report</article-title>
          ,” in Requirements Engineering:
          <article-title>Foundation for Software Quality</article-title>
          . Springer,
          <year>2011</year>
          , pp.
          <fpage>19</fpage>
          -
          <lpage>33</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          [4]
          <string-name>
            <given-names>L. M.</given-names>
            <surname>Hilty</surname>
          </string-name>
          and
          <string-name>
            <given-names>B.</given-names>
            <surname>Aebischer</surname>
          </string-name>
          , “
          <article-title>ICT for sustainability: An emerging research field,” in ICT Innovations for Sustainability</article-title>
          . Springer,
          <year>2015</year>
          , pp.
          <fpage>3</fpage>
          -
          <lpage>36</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          [5]
          <string-name>
            <given-names>F.</given-names>
            <surname>Berkhout</surname>
          </string-name>
          and
          <string-name>
            <given-names>J.</given-names>
            <surname>Hertin</surname>
          </string-name>
          , “
          <article-title>Impacts of information and communication technologies on environmental sustainability: Speculations and evidence,” Report to the OECD</article-title>
          , vol.
          <volume>21</volume>
          ,
          <year>2001</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          [6]
          <string-name>
            <given-names>T.</given-names>
            <surname>Johann</surname>
          </string-name>
          and
          <string-name>
            <given-names>W.</given-names>
            <surname>Maalej</surname>
          </string-name>
          , “
          <article-title>The social dimension of sustainability in requirements engineering</article-title>
          ,” in 2nd International Workshop on RE4SuSy, vol.
          <volume>995</volume>
          .
          <string-name>
            <surname>CEUR-WS</surname>
          </string-name>
          ,
          <year>2013</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          [7]
          <string-name>
            <given-names>B.</given-names>
            <surname>Penzenstadler</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Raturi</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Richardson</surname>
          </string-name>
          , and
          <string-name>
            <given-names>B.</given-names>
            <surname>Tomlinson</surname>
          </string-name>
          , “
          <article-title>Safety, security, now sustainability: the nonfunctional requirement for the 21st century</article-title>
          ,
          <source>” IEEE Software</source>
          , vol.
          <volume>31</volume>
          , no.
          <issue>3</issue>
          , pp.
          <fpage>40</fpage>
          -
          <lpage>47</lpage>
          ,
          <year>2014</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          [8]
          <string-name>
            <given-names>K.</given-names>
            <surname>Roher</surname>
          </string-name>
          and
          <string-name>
            <given-names>D.</given-names>
            <surname>Richardson</surname>
          </string-name>
          , “
          <article-title>A proposed recommender system for eliciting software sustainability requirements,” in 2nd International Workshop on User Evaluations for Software Engineering Researchers</article-title>
          . IEEE,
          <year>2013</year>
          , pp.
          <fpage>16</fpage>
          -
          <lpage>19</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          [9]
          <string-name>
            <given-names>N.</given-names>
            <surname>Amsel</surname>
          </string-name>
          ,
          <string-name>
            <given-names>Z.</given-names>
            <surname>Ibrahim</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Malik</surname>
          </string-name>
          , and
          <string-name>
            <given-names>B.</given-names>
            <surname>Tomlinson</surname>
          </string-name>
          , “
          <article-title>Toward sustainable software engineering</article-title>
          ,
          <source>” in 33rd International Conference on Software Engineering. ACM</source>
          ,
          <year>2011</year>
          , pp.
          <fpage>976</fpage>
          -
          <lpage>979</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          [10]
          <string-name>
            <given-names>A.</given-names>
            <surname>Raturi</surname>
          </string-name>
          ,
          <string-name>
            <given-names>B.</given-names>
            <surname>Penzenstadler</surname>
          </string-name>
          ,
          <string-name>
            <given-names>B.</given-names>
            <surname>Tomlinson</surname>
          </string-name>
          , and
          <string-name>
            <given-names>D.</given-names>
            <surname>Richardson</surname>
          </string-name>
          , “
          <article-title>Developing a sustainability non-functional requirements framework</article-title>
          ,
          <source>” in 3rd International Workshop on Green and Sustainable Software. ACM</source>
          ,
          <year>2014</year>
          , pp.
          <fpage>1</fpage>
          -
          <lpage>8</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          [11] “ISO 14040:
          <year>2006</year>
          .
          <article-title>Environmental management - Life cycle assessment - Principles and framework</article-title>
          ,” International Organization for Standardization,
          <year>2006</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          <source>[12] UN World Commission on Environment and Development</source>
          , “
          <article-title>Our common future</article-title>
          ,” pp.
          <fpage>1</fpage>
          -
          <lpage>300</lpage>
          ,
          <year>1987</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          [13]
          <string-name>
            <given-names>M.</given-names>
            <surname>Christen</surname>
          </string-name>
          , “
          <article-title>A theory of the good for a conception of sustainability,”</article-title>
          <source>in 16th Annual International Sustainable Development Research Conference</source>
          ,
          <year>2010</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          [14]
          <string-name>
            <given-names>C.</given-names>
            <surname>Becker</surname>
          </string-name>
          , “
          <article-title>Sustainability and longevity: Two sides of the same quality?</article-title>
          ”
          <source>in 3rd International Workshop on RE4SuSy</source>
          , vol.
          <volume>1216</volume>
          .
          <string-name>
            <surname>CEUR-WS</surname>
          </string-name>
          ,
          <year>2014</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref15">
        <mixed-citation>
          [15]
          <string-name>
            <given-names>M.</given-names>
            <surname>Glinz</surname>
          </string-name>
          , “On
          <string-name>
            <surname>Non-Functional</surname>
            <given-names>Requirements</given-names>
          </string-name>
          ,” in 15th IEEE International Requirements Engineering Conference. IEEE,
          <year>2007</year>
          , pp.
          <fpage>21</fpage>
          -
          <lpage>26</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref16">
        <mixed-citation>
          [16]
          <string-name>
            <given-names>J.</given-names>
            <surname>Cabot</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Easterbrook</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Horkoff</surname>
          </string-name>
          ,
          <string-name>
            <given-names>L.</given-names>
            <surname>Lessard</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Liaskos</surname>
          </string-name>
          , and
          <string-name>
            <given-names>J.</given-names>
            <surname>Mazón</surname>
          </string-name>
          , “
          <article-title>Integrating sustainability in decision-making processes: A modelling strategy</article-title>
          ,
          <source>” in 31st International Conference on Software Engineering. IEEE</source>
          ,
          <year>2009</year>
          , pp.
          <fpage>207</fpage>
          -
          <lpage>210</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref17">
        <mixed-citation>
          [17]
          <string-name>
            <given-names>B.</given-names>
            <surname>Penzenstadler</surname>
          </string-name>
          and
          <string-name>
            <given-names>H.</given-names>
            <surname>Femmer</surname>
          </string-name>
          , “
          <article-title>A generic model for sustainability with process-and product-specific instances</article-title>
          ,” in Workshop on Green in/by Software Engineering (GIBSE
          <year>2013</year>
          ). ACM,
          <year>2013</year>
          , pp.
          <fpage>3</fpage>
          -
          <lpage>8</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref18">
        <mixed-citation>
          [18]
          <string-name>
            <given-names>C.</given-names>
            <surname>Bomfim</surname>
          </string-name>
          ,
          <string-name>
            <given-names>W.</given-names>
            <surname>Nunes</surname>
          </string-name>
          ,
          <string-name>
            <given-names>L.</given-names>
            <surname>Duboc</surname>
          </string-name>
          , and
          <string-name>
            <given-names>M.</given-names>
            <surname>Schots</surname>
          </string-name>
          , “
          <article-title>Modelling sustainability in a procurement system: An experience report</article-title>
          ,” in 22nd International Requirements Engineering Conference. IEEE,
          <year>2014</year>
          , pp.
          <fpage>402</fpage>
          -
          <lpage>411</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref19">
        <mixed-citation>
          [19]
          <string-name>
            <given-names>K.</given-names>
            <surname>Roher</surname>
          </string-name>
          and
          <string-name>
            <given-names>D.</given-names>
            <surname>Richardson</surname>
          </string-name>
          , “Sustainability requirement patterns,” in 3rd International Workshop on Requirements Patterns. IEEE,
          <year>2013</year>
          , pp.
          <fpage>8</fpage>
          -
          <lpage>11</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref20">
        <mixed-citation>
          [20]
          <string-name>
            <given-names>X.</given-names>
            <surname>Wang</surname>
          </string-name>
          , “
          <article-title>A comprehensive decision making model for the evaluation of green operations initiatives,” Technological Forecasting and Social Change</article-title>
          , vol.
          <volume>95</volume>
          , pp.
          <fpage>191</fpage>
          -
          <lpage>207</lpage>
          ,
          <year>2015</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref21">
        <mixed-citation>
          [21]
          <string-name>
            <given-names>I.</given-names>
            <surname>Benbasat</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D. K.</given-names>
            <surname>Goldstein</surname>
          </string-name>
          , and
          <string-name>
            <given-names>M.</given-names>
            <surname>Mead</surname>
          </string-name>
          , “
          <article-title>The case research strategy in studies of information systems</article-title>
          ,
          <source>” MIS Quarterly</source>
          , vol.
          <volume>11</volume>
          , pp.
          <fpage>369</fpage>
          -
          <lpage>386</lpage>
          ,
          <year>1987</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref22">
        <mixed-citation>
          [22]
          <string-name>
            <surname>R. B. Svensson</surname>
            ,
            <given-names>T.</given-names>
          </string-name>
          <string-name>
            <surname>Olsson</surname>
            , and
            <given-names>B.</given-names>
          </string-name>
          <string-name>
            <surname>Regnell</surname>
          </string-name>
          , “
          <article-title>An investigation of how quality requirements are specified in industrial practice</article-title>
          ,
          <source>” Information and Software Technology</source>
          , vol.
          <volume>55</volume>
          , no.
          <issue>7</issue>
          , pp.
          <fpage>1224</fpage>
          -
          <lpage>1236</lpage>
          ,
          <year>2013</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref23">
        <mixed-citation>
          [23]
          <string-name>
            <given-names>S.</given-names>
            <surname>Easterbrook</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Singer</surname>
          </string-name>
          , M.
          <article-title>-</article-title>
          <string-name>
            <surname>A. Storey</surname>
            , and
            <given-names>D.</given-names>
          </string-name>
          <string-name>
            <surname>Damian</surname>
          </string-name>
          , “
          <article-title>Selecting empirical methods for software engineering research,” in Guide to Advanced Empirical Software Engineering</article-title>
          . Springer,
          <year>2008</year>
          , pp.
          <fpage>285</fpage>
          -
          <lpage>311</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref24">
        <mixed-citation>
          [24]
          <string-name>
            <given-names>B. A.</given-names>
            <surname>Kitchenham</surname>
          </string-name>
          and
          <string-name>
            <given-names>S. L.</given-names>
            <surname>Pfleeger</surname>
          </string-name>
          , “
          <article-title>Personal opinion surveys,” in Guide to Advanced Empirical Software Engineering</article-title>
          . Springer,
          <year>2008</year>
          , pp.
          <fpage>63</fpage>
          -
          <lpage>92</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref25">
        <mixed-citation>
          [25]
          <string-name>
            <given-names>H.</given-names>
            <surname>Beyer</surname>
          </string-name>
          and
          <string-name>
            <given-names>K.</given-names>
            <surname>Holtzblatt</surname>
          </string-name>
          ,
          <article-title>Contextual design: defining customercentered systems</article-title>
          . Morgan Kaufmann,
          <year>1998</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref26">
        <mixed-citation>
          [26]
          <string-name>
            <given-names>M. Z.</given-names>
            <surname>Huber</surname>
          </string-name>
          and
          <string-name>
            <given-names>L. M.</given-names>
            <surname>Hilty</surname>
          </string-name>
          , “
          <article-title>Gamification and sustainable consumption: overcoming the limitations of persuasive technologies,” in ICT Innovations for Sustainability</article-title>
          . Springer,
          <year>2015</year>
          , pp.
          <fpage>367</fpage>
          -
          <lpage>385</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref27">
        <mixed-citation>
          [27]
          <string-name>
            <surname>C. B. Seaman</surname>
          </string-name>
          , “
          <article-title>Qualitative methods in empirical studies of software engineering</article-title>
          ,
          <source>” IEEE Transactions on Software Engineering</source>
          , vol.
          <volume>25</volume>
          , no.
          <issue>4</issue>
          , pp.
          <fpage>557</fpage>
          -
          <lpage>572</lpage>
          ,
          <year>1999</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref28">
        <mixed-citation>
          [28]
          <string-name>
            <given-names>J.</given-names>
            <surname>Singer</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S. E.</given-names>
            <surname>Sim</surname>
          </string-name>
          , and T. C. Lethbridge, “
          <article-title>Software engineering data collection for field studies,” in Guide to Advanced Empirical Software Engineering</article-title>
          . Springer,
          <year>2008</year>
          , pp.
          <fpage>9</fpage>
          -
          <lpage>34</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref29">
        <mixed-citation>
          [29]
          <string-name>
            <given-names>C. E.</given-names>
            <surname>Osgood</surname>
          </string-name>
          ,
          <string-name>
            <given-names>G. J.</given-names>
            <surname>Suci</surname>
          </string-name>
          , and
          <string-name>
            <given-names>P. H.</given-names>
            <surname>Tannenbaum</surname>
          </string-name>
          ,
          <article-title>The measurement of meaning</article-title>
          . University of Illinois Press,
          <year>1964</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref30">
        <mixed-citation>
          [30]
          <string-name>
            <given-names>R.</given-names>
            <surname>Likert</surname>
          </string-name>
          , “
          <article-title>A method of constructing an attitude scale,” in Scaling: A Sourcebook for Behavioral Scientists</article-title>
          ,
          <year>1974</year>
          , pp.
          <fpage>233</fpage>
          -
          <lpage>243</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref31">
        <mixed-citation>
          [31]
          <string-name>
            <given-names>E. P.</given-names>
            <surname>Cox</surname>
          </string-name>
          <string-name>
            <surname>III</surname>
          </string-name>
          , “
          <article-title>The optimal number of response alternatives for a scale: A review</article-title>
          ,
          <source>” Journal of Marketing Research</source>
          , vol.
          <volume>17</volume>
          , pp.
          <fpage>407</fpage>
          -
          <lpage>422</lpage>
          ,
          <year>1980</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref32">
        <mixed-citation>
          [32]
          <string-name>
            <given-names>C. H.</given-names>
            <surname>Yu</surname>
          </string-name>
          , “
          <article-title>Abduction? deduction? induction? is there a logic of exploratory data analysis?” in Annual Meeting of American Educational Research Association</article-title>
          . ERIC Document Reproduction Service No.
          <source>ED 376 173</source>
          ,
          <year>1994</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref33">
        <mixed-citation>
          [33]
          <string-name>
            <surname>C. B. Seaman</surname>
          </string-name>
          , “
          <article-title>Qualitative methods,” in Guide to Advanced Empirical Software Engineering</article-title>
          . Springer,
          <year>2008</year>
          , pp.
          <fpage>35</fpage>
          -
          <lpage>62</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref34">
        <mixed-citation>
          [34]
          <string-name>
            <given-names>C.</given-names>
            <surname>Wohlin</surname>
          </string-name>
          ,
          <string-name>
            <given-names>P.</given-names>
            <surname>Runeson</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Höst</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M. C.</given-names>
            <surname>Ohlsson</surname>
          </string-name>
          ,
          <string-name>
            <given-names>B.</given-names>
            <surname>Regnell</surname>
          </string-name>
          ,
          <article-title>and</article-title>
          <string-name>
            <given-names>A.</given-names>
            <surname>Wesslén</surname>
          </string-name>
          , Experimentation in Software Engineering. Springer (originally published by Kluwer in
          <year>2000</year>
          ),
          <year>2012</year>
          .
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>