=Paper= {{Paper |id=Vol-1944/paper4 |storemode=property |title=Mind the Chasm: A FishEye Lens View of Sustainable Software Engineering |pdfUrl=https://ceur-ws.org/Vol-1944/paper4.pdf |volume=Vol-1944 |authors=Madeleine L. Gibson,Colin C. Venters,Leticia Duboc,Stefanie Betz,Ruzanna Chitchyan,Maria Palacin-Silva,Birgit Penzenstadler,Norbert Seyff }} ==Mind the Chasm: A FishEye Lens View of Sustainable Software Engineering== https://ceur-ws.org/Vol-1944/paper4.pdf
                   Mind the Chasm: A FishEye Lens View
                    of Sustainable Software Engineering
                          in UK Higher Education
     Madeleine L. Gibson                          Colin C. Venters                             Leticia Duboc                                  Stefanie Betz
  University of Huddersfield University of Huddersfield State Univ. of Rio de Janeiro Karlsruhe Insitute of Technology
      Huddersfield, UK            Huddersfield, UK          Rio de Janeiro, Brazil           Karlsruhe, Germany
 U1364186@unimail.hud.ac.uk     c.venters@hud.ac.uk           leticia@ime.uerj.br           stefanie.betz@kit.edu


      Ruzanna Chitchyan                         Maria Palacin-Silva                       Birgit Penzenstadler                                Norbert Seyff
     University of Leicester                           LUT                      CSU Long Beach         FHNW and University of Zurich
         Leicester, UK                        Lappeenranta, Finland         Long Beach, CA, USA           Windisch, Switzerland
     rc256@leicester.ac.uk                   maria.palacin.silva@lut.fi birgit.penzenstadler@csulb.edu    norbert.seyff@fhnw.ch




   Abstract—Requirements that express the needs of all stake-                             a paradigm shift in research towards large-scale computational
holders and cover the key aspects of a software system (such                              science and engineering, and data intensive science [5], [6].
as those addressing sustainability) are critical to the system’s                          Software’s increasing importance in the field of research, has
successful development and adoption. For practitioners who want
to develop sustainable software-intensive systems, it is also argued                      led for calls for it to be classified as a first-class, experimental
that software requirements are the key leverage point.                                    scientific instrument [7] based on the maxim, if your software
   But what do software developers know about broader sustain-                            is incorrect, so will your science [8]. However, it is suggested
ability and sustainability requirements in particular? As part of                         that in terms of structure, content, and functionality, software
the University of Huddersfield’s ’Student as a Researcher’ initia-                        systems are the most complex artifacts ever created [9]. As
tive1 , this paper provides a FishEye2 Lens View on how novice
software developers who design and develop software systems at                            such, software designers are responsible for the long-term
software companies relate to the notions of sustainability and                            consequences of their designs [10].
sustainability requirements. The study has found that although                               The vision of sustainable software engineering is based on
sustainability is valued highly by the novice software developers,                        the idea of delivering software systems, which are designed
the concept of sustainability and sustainability requirements are                         and developed in a way that the consideration of the different
not fully understood. This lack of knowledge in the software
industry and organizations must be addressed to enable delivery                           sustainability dimensions and orders of effects are reflected
of truly sustainability enabling software systems.                                        in the system. Although there is no clear definition of the
   Index Terms—Karlskrona manifesto, requirements engineer-                               term ’sustainability requirement’ [11], the bespoke reflection
ing, software engineering, software requirements, sustainability,                         and discussion is often based on requirements that align
sustainability design, sustainability requirements, sustainable                           with specific dimensions of sustainability. This highlights the
software
                                                                                          importance of requirements engineering as a central aspect
                                                                                          of sustainable software engineering. Moreover, it is argued
                             I. I NTRODUCTION
                                                                                          that requirements are the key to sustainability, as it is during
   Post-industrial societies are highly dependent on complex                              the requirement engineering phase when the foundations of a
software systems that underlie almost every aspect of daily                               systems are laid [12].
living from communication, entertainment, energy, commerce,                                  The goal of the work presented in this paper is to better
transportation, finance, governance, health care, as well as                              understand the perception of sustainable software engineer-
defense and security [2], [3], [4]. In addition, software also                            ing among UK students enrolled on computing degree pro-
now plays a critical role in the advancement of knowledge with                            grammes and young software developers in industry, which
                                                                                          we will collectively refer to as novice software engineers
   1 Student as a Researcher is ’a pedagogic approach to supporting students in
                                                                                          in the reminder of this paper. More specifically, the paper
their engagement with undergraduate research within and/or beyond the formal
curriculum with the aim of furthering their own knowledge and understanding,              aims to find out what these novice developers think about
and in some cases contributing to, the broader knowledge base of their                    sustainability requirements and what they may know about
discipline. The term is used to describe a pedagogic approach, rather than                it. The remainder of this paper is structured as follows: In
the students themselves’ [1].
   2 A Fisheye lens is a special type of lens that shows a distorted view of the          section II, we describe relevant background research, while
world.                                                                                    section III gives an overview of the methodology used for


 Copyright © 2017 for the individual papers by the papers' authors. Copying permitted for private and academic purposes. This volume is published and copyrighted by its editors.
the study. In section IV, we present the results of this work,          There exist interdependencies between these dimensions
with several relevant issues discussed in section IV. The paper      including trade-offs and synergies. Moreover, the impact on
concludes in Section V, with recommendations for next steps          sustainability in these five dimensions manifests in three
toward a more comprehensive and consistent perspective on            different orders of effect [12], [19], defined as follows:
sustainability requirements and their relationship to the field of     • First order effects appear when software systems are built
software engineering, and the role of education in facilitating          and used for their primary purpose, e.g. the work time
a programme of sustainable software engineering.                         used for implementation of Youtube;
                                                                       • Second order effects appear when the use of the system
                      II. BACKGROUND                                     over some time induces new types of behavior or expec-
A. Sustainability and Software                                           tations for the previous system, e.g. the energy waste due
                                                                         to letting Youtube videos play in the background, while
   Derived from the Latin word sustinere, the Oxford English
                                                                         only listening to the music;
Dictionary [13] defines sustainability as ‘the quality of being
                                                                       • Finally, third order effects appear due to a large-scale,
sustained’, where sustained can be defined as ‘capable of be-
                                                                         longer term use of the system, e.g. as most people use
ing endured’ and ‘capable of being maintained’. This suggests
                                                                         YouTube for music, professional music record producers
that longevity and the ability to maintain are key factors at the
                                                                         and sellers gradually disappear as domain actors (which is
heart of understanding sustainability. As a part of the concept
                                                                         a clear detrimental effect on the economic sustainability
of sustainable development, the Brundtland commission [14],
                                                                         of these businesses), making self-production and free
defined the term as ‘meeting the needs of the present without
                                                                         availability of music on Youtube a new social norm
compromising the ability of future generations to meet their
                                                                         (which democratises the music market for the musicians,
own needs’. The word ‘need’ is central to this definition and
                                                                         and increases the variety of available music for con-
includes a dimension of ‘time’, present and future.
                                                                         sumers, positively affecting individual and social sustain-
   In the last decade the concept of sustainability has emerged
                                                                         ability).
as a topic of interest in a range of different areas in the field
of computing including software engineering (SE) [15] and               While there have been a number of contributions to formal-
requirements engineering (RE) [11]. In relation to software and      ize a definition of software sustainability [20], [15], [16], [18],
software systems, there are primarily two distinct viewpoints        [21], consensus on what sustainability means in the field of
that make up the topic area around software and sustainability:      software and requirements engineering is still emerging [22],
sustainable software and software engineering for sustainabil-       [23]. The term is increasingly described in the literature as a
ity (SE4S), where the former is concerned with the principles,       first-class, non-functional requirement or software quality [24],
practices, and process that contribute to software endurance i.e.    [25], [11]. However, research is required to confirm or refute
technical sustainability [16], and the latter focuses on software    this position.
systems to support one or more dimensions of sustainability,            Furthermore, Ramsey [26] argues that the fundamental
concerning issues outside the software systems itself [17].          problem with definitions of sustainability is the definitional
The Karlskrona Manifesto [10] recognizes both viewpoints             approach itself and not just with any particular definition.
on sustainability as an emerging concern of central relevance        It is suggested that definitions do not lead to clarity about
and suggests to view sustainability as a construct across the        the meaning of sustainability because the meaning of the
five dimensions of sustainability: environmental, economic,          component terms, as well as the meaning of the general
individual, social and technical [12], [18]. These dimensions        concept, presupposes the existence of a set of social structures
are defined as follows:                                              within which the activity of defining makes sense. As such,
                                                                     any attempt to impose interpretations without recognizing the
   • The economic dimension refers to assets, capital, and
                                                                     necessity and importance of those structures are doomed to
      added value including for example profitability and cap-
                                                                     failure.
      ital investment, etc;
   • The environmental dimension refers to the long term
                                                                        As a result, the concept of sustainability requires context
      effects of human activities on natural ecosystems and          (such as that proposed by Tainter [27]), [social] structure
      resources;                                                     [26], and the simultaneous consideration of several interrelated
   • The individual dimension refers to the individual well-
                                                                     dimensions of sustainability [12]. In addition, rather than
      being including for example freedom, self-respect, edu-        seeking broad conformity of definitions, the aim should be
      cation etc;                                                    to clarify how the terms are used by different communities in
   • The social dimension refers to the interaction between
                                                                     order to have a shared understanding [28].
      individuals and the societal constructs based on this in-
                                                                     B. Sustainability Studies in Software Engineering
      teraction and constituting a society including for example
      trust, communication, employment, democracy, etc;                 A number of recent studies have been conducted to under-
   • The technical dimension refers to the concepts of               stand how sustainability is perceived in the practice of software
      longevity of man-made systems, and infrastructure and          engineering and how sustainability can become an inherent
      their adequate evolution.                                      part of software engineering practice.
   Chitchyan et. al., [29] explored requirements engineering        through analysis of 11 case studies. The overall results suggest
practitioners’ perceptions and attitudes towards sustainability     that technical and economic sustainability are the primary
through interviews. The results revealed that software practi-      focus in current SPLE practice, with social sustainability
tioners tend to have a narrow understanding of the concept          issues, where they relate to organisations, also addressed.
of sustainability with a focus on environmental and economic        However, environmental and individual sustainability concerns
sustainability. Their organizations also showed limited aware-      were less prominent.
ness of the potential opportunities and benefits for engineering        Overall, these studies suggest that practitioners have a
sustainability in or through software. Notably, several inter-      narrow perception of sustainability and that sustainability is
viewees explicitly considered sustainability as a separate field    not a major concern in their current projects. However, more
from that of software engineering.                                  elaborate concepts and a more stable theoretical background
   Manotas et. al., [30] investigated what software practitioners   regarding the body of knowledge about sustainable software
from ABB, Google, IBM, and Microsoft thought about energy           engineering is emerging, which might also influence the
usage in relation to requirements, design, construction, testing,   perception of suitability in software engineering education.
and maintenance of their software. Their results suggest that       Therefore, we could expect to see a higher degree of awareness
software engineering practitioners are concerned about energy       regarding sustainability issue when it comes to undergraduate
when they build applications; however, it was suggested that        and graduate students, but there is no related work exploring
they are not as successful as they could be because they            this issue. The research presented in this paper contributes to
lack the necessary information and support infrastructure. In       filling this void.
addition, the results also highlighted that energy-usage is often
stated in terms other than energy usage being more often            C. Emergence of Sustainability Requirements
desires rather than specific targets, which focus on idle time         As the concept of sustainability has started to be discussed
and are difficult to specify directly.                              in software engineering, so has the concept of sustainability
   Kasurinen, Palacin-Silva and Vanhala [31] investigated the       requirements. Becker et. al., [12] argue that the critical role
views of professional game developers on sustainability and         that software plays in society demands a paradigm shift in
Green IT. The results of their study highlighted that Game          the mindset of software engineering and that the focus of
development organizations are not likely to consider eco-           this shift begins in requirements engineering. A number of
impact factors such as code reusability, energy-efficient pro-      approaches have been proposed for sustainability requirements
graming, social awareness impact, reuse or repurposing old          elicitation [34], [35], codification [36], modeling [37], [38],
hardware components or marketing materials or support to            and management [18]. For example Penzenstadler and Femmer
legacy systems in their daily work. If anything at all, code        [18] proposed a goal modelling based approach to capture
reusability was the most regarded factor due to efficiency rea-     sustainability requirements using a generic reference model.
sons. This suggests that the game developers do not recognize       Betz [39] presents a business process based view to modelling
what sustainability means in their industry. Their organiza-        sustainability requirements. While Makropoulos [40] analyses
tions, as a whole, do not practice sustainability-conducive         the specific case of sustainable urban water management,
activities either. The authors attributed this to the lack of       proposing a general model of sustainability that needs to be
a common definition of sustainability in the field as a way         contextualized to assess sustainability. As far as we can ascer-
of understanding what constituted eco-impact factors; a view        tain, there exists no formal definition of the term sustainability
supported by Penzenstadler [18]. In addition, it was also           requirement. In most instances, a sustainability requirement is
suggested that a sustainability body of knowledge for software      defined in terms of a non-functional requirement or software
engineering could provide specific guidance with regards to         quality and aligned with one or more of the dimensions of
addressing sustainability aspects from a software development       sustainability [37], [34], [36], [41] based on characteristics
perspective.                                                        defined in ISO/IEC 25010 [42]: functional suitability, reliabil-
   Groher and Weinreich [32] investigated how sustainability        ity, performance, efficiency, usability, security, compatibility,
is perceived by software engineering professionals in order         maintainability, and portability. Roher and Richardson [35]
to understand how sustainability is currently dealt with in         defined a sustainability requirements as ‘requirements that may
software development projects in practice. The results of the       be used to specify system behavior (e.g. requirements that will
study revealed that practitioners primarily relate sustainability   reduce a system’s energy consumption) as well as to influence
to technical sustainability with a strong focus on maintain-        the users’ behavior (e.g. the system incentivizes sustainable
ability and extensibility of the software systems. Economic         actions)’. In contrast, Huber, Hilty, and Glinz [43] in their
sustainability was an indirect consideration related to influ-      investigation into the requirements of a decision support
encing factors such as the market and customers. In contrast        system (DSS) suggested that a sustainability requirement is a
to Chitchyan et. al., [29], environmental sustainability was not    ‘requirement for a sustainable software system which concerns
mentioned at all by the study participants.                         sustainability’. They argue that sustainability requirements
   Chitchyan, Groher and Noppen [33] investigated what sus-         cannot simply be reduced to quality aspects and should be
tainability means to the field of software engineering within       related to positive effects suggesting that when positive en-
the context of Software Product Line Engineering (SPLE)             abling effects are taken into consideration the requirements
for a software system change under the lens of sustainable            •   Former students were required to hold a Bachelors of
development. While the latter definition is overly simplistic it          Science (BSc.) or Masters (MEng, MSc.) degree in one
implicitly suggests that requirements can be viewed through a             of the above listed areas;
multi-perspective lens of sustainability. The former definition        • Employed in the software industry for a minimum of six
essentially points towards a subset of functional requirements            months;
and towards user influence. As a result, neither definition            • Worked or are currently working in the software industry
provides any insight into how sustainability requirements                 as part of their industrial placement year3 ;
differ from existing taxonomic classifications of requirements         • Elicited software requirements as part of their respective
[44], [45]. However, Venters et. al. [11] revealed that the               projects.
term sustainability requirement is used ambiguously and con-           The study included eight participants of which, seven par-
tains significant variations across the different application and   ticipants were male and one was female. Four are currently
scientific discipline domains. Their study suggests, that the       working in the software industry as part of their industrial
dimensions of sustainability and its time effects could help to     placement year. Three were former students currently em-
see which requirements pertain to sustainability of the software    ployed in the software industry. One was enrolled on the
system and its wider impact.                                        MEng. degree programme in Software Engineering. Their age
                     III. M ETHODOLOGY                              range was between 20-29 years and they were associated with
                                                                    one of the following institutions:
   The principal aim of this study is to explore the knowledge                              4
                                                                       • LNT Software : a small to medium sized Enterprise
of novice software developers with respect to sustainability,             (SME) software production company dedicated to the de-
sustainability requirements, and their relationship to software           sign and development of a range of management software
engineering principles and practices. Its objective is two-               systems used in care homes across the UK;
folded: to learn about the perspectives and ideas of novice            • School of Computing and Engineering, University of
software developers in relation to sustainability and sustain-            Huddersfield5 : a public university located in Hudders-
ability requirements, and to highlight whether sustainability is          field, West Yorkshire, UK;
taught within the software engineering education. Therefore,                                                6
                                                                       • Canalside Studios Game Design : an independent game
the study considered the following research questions:                    studio, within the School of Computing and Engineering
  RQ1 What are the opinions and expectations of novice                    at the University of Huddersfield.
      software developers on sustainability and sustainabil-           3) Data Collection: The data collection was undertaken
      ity requirements?                                             through in-person, semi-structured interview, which lasted
  RQ2 What do novice software developers know about                 approximately thirty minutes and covered four key areas:
      sustainability (within software engineering) when                1) Personal background, including demographics, software
      developing software systems?                                          industry experience, and education level;
                                                                       2) Sustainability, including their understanding of the term
A. Study Design
                                                                            and concrete actions towards sustainability in both pri-
   A qualitative research approach was chosen to answer the                 vate and professional lives;
above research questions using semi-structured interviews,             3) Software engineering principles and practices, normally
which included open and closed questions to elicit the data.                applied in their work and their potential to contemplate
This approach allows the classification of the data into con-               sustainability;
cepts and themes in order to explain a particular phenomena            4) Sustainability requirements, covering their understand-
and to gain a deeper understanding of the underlying reasons,               ing of functional and non-functional requirements, as
opinions, and motivations [46], [47].                                       well as sustainability requirements.
   1) Planning: At the planning stage, the interview questions         4) Data Analysis: The qualitative content analysis method
were designed collaboratively by all authors. The study was         [48] was used to extract views and perceptions on sustain-
piloted with one interviewee to validate the clarity of questions   ability, sustainability requirements, and software engineering
and the interview structure resulting in no major changes.          principles and practice from these interview transcripts. The
   2) Interviewees: The subjects were selected from current         interviews were fully transcribed and analyzed using Saturate
and former students from the School of Computing and En-            [49]; a qualitative data analysis tool. A codebook was created
gineering at the University of Huddersfield, using the contact      by the first two authors of the paper in order for the data to be
network of the first two authors of this paper. The criteria for    classified and trends observed. The initial set of codes were
selection was as follows:                                              3 All undergraduate programs in the School of Computing and Engineering
  •   Current students were required to be enrolled on a Bach-      at the University of Huddersfield include an optional year in industry:
      elors of Science (BSc.) or Masters (MEng, MSc.) degree        https://www.hud.ac.uk/ce/placements/
                                                                       4 LNT Software: www.lntsoftware.com
      programme in Computing, Computer Science, Games                  5 School of Computing and Engineering, University of Huddersfield:
      Design, Game Programming, Information Systems or              https://www.hud.ac.uk/ce/
      Software Engineering;                                            6 Canalside Studios Game Design: http://www.canalsidestudios.com
created by the first coder and was updated with each following      terms in relation to sustainability generally had a good un-
coding activity. The initial codebook, as well as the updates,      derstanding of the environmental, economic and technical
were discussed and agreed upon by all co-authors of this paper.     dimensions of sustainability but lacked any real understanding
The first two authors read each of the interviews and coded         with regard to the individual and social dimensions; these were
the text with conceptual categories relevant to sustainability      unheard of by seven of the participants. The other participant
perceptions, as well as peer-reviewed each other’s work. The        stated that they had not considered sustainability in relation
interview guide and the codebook are available at [50].             to the five dimensions. This is in itself surprising especially
                                                                    with regards to the prominence of environmental sustainability
B. Limitations of the Study                                         in post-industrial societies particularly around recycling and
   As is normally the case with qualitative research, the results   energy conservation.
cannot be generalized to the wider population of software              Four interviewees (Scott, Claire, Steven and Thomas) stated
developers. This is both due to the limitations of the approach     that they had considered environmental sustainability, citing is-
and the limited number of subjects [51]. In addition, this study    sues related to the impact of the environment including global
focused solely on novice software developers, as a means to         warming, deforestation, resource depletion and farming, as
also evaluate the coverage of sustainability within software        well as ways to mitigate that impact in terms of moving
engineering education. As a result, participants may not have       towards renewable energy sources such as wind and tidal,
had enough industry experience to provide valuable insights         and recycling. However, Steven also highlighted that envi-
into sustainability with software engineering and sustainability    ronmental sustainability “doesn’t need to be natural, it could
requirements. Furthermore, the fact that all participants at-       be artificial such as this office space” emphasizing that the
tended the same university, also limits the conclusions to that     environments we work in also need to consider sustainability.
particular institution. However, the results corroborate with the   In addition, Thomas related environmental sustainability to
findings from other similar studies [29], [31], [32].               hardware and server capacity stating that “if the servers went
                                                                    down, the software’s environment must also be sustainable”,
                    IV. S TUDY F INDINGS                            indirectly acknowledging the relevance of the wider notion of
                                                                    the systems dependability for sustainability.
   The main findings of this qualitative study fall into three         Four interviewees (Steven, Thomas, Ryan, James) also
categories, which are discussed in the following sections. Each     stated that they had considered economical sustainability, three
of the individual interviewees is referenced by a fictitious name   of whom referred to it as ensuring business value, capital
to ensure anonymity.                                                growth and investment, and financial operations. For example,
                                                                    James suggested that in terms of economic sustainability “I
A. Sustainability Findings
                                                                    don’t think they’ve made as much money as they have put into
   The first set of questions were intended to uncover the par-     this [Canalside Studios], so really, they are paying for the
ticipants’ understanding of and commitment to sustainability.       students to get experience really”.
   What is your understanding of the concept of Sustainabil-           Finally, four interviewees (Steven, Claire, Mark and Ryan)
ity? All interviewees related sustainability to the concept of      stated that they had considered technical sustainability, which
endurance over time. For example, Scott stated that sustain-        covered both the software and hardware aspects of software
ability is “something that can be sustained effectively over a      systems. Steven stated that technical sustainability was “re-
long or short period of time”. In addition, two interviewees        lated to the ability to build a software product that is built
suggested that sustainability could also be related to tech-        in a way so that it does not die”. Similarly, Claire and Mark
nical aspect of sustainability including software metrics and       highlighted the importance of maintainability and extendability
hardware. Thomas stated that sustainability was also related        of the software systems. Claire emphasized the importance of
to the “metrics used to quantify how sustainable a piece of         “the ability to maintain and evolve software systems over a
software is” both through the “development phase and after          prolonged period of time”. Mark explained that “I program in
its hand-off to a client”. Jamie suggested that sustainability      a way that’s easily maintainable for people other than myself,
was related to the longevity of resources stating that there        I make short methods that are recognizable so we minimize
are issues with “hardware sustainability” particularly with         the amount of lines we use”. Ryan suggested that technical
advances in hardware driving the game industry. Ryan said that      sustainability was related to effectively utilizing resources,
the concept should be considered in relation to the environment     explaining that “technical sustainability is not using more
in a more direct manner, stating that sustainability is also        resources than you have, such as computer power or man
related to “using less or an equal amount of resources that         power”.
are being produced at the same time”.                                   What do you do for sustainability in your daily private
   Have you ever considered the following sustainability            life? The most common answers amongst participants were
dimensions: Environmental, Economic, Individual, Social             related to [environmental] sustainability. Ben, Jamie, Scott,
or Technical? When asked whether they had ever considered           Claire, Steven and Thomas articulated tasks such as switching
sustainability in relation to the five sustainability dimensions,   lights off, saving energy, and recycling. However, Ben, Jamie
those participants who understood the differentiation of the        and Scott also emphasized the importance of saving and
investment in relation to ensure economic stability thereby                      DRY11 , KISS12 , and best software engineering practice.
linking activities related to economic resource usage to [eco-                      What are the software engineering principles you apply
nomic] sustainability. However, Mark stated, “I don’t think I                    that are related to sustainability in software systems? Similar
do sustainable, I rarely recycle or take consideration of the                    to the responses to the previous question on software engi-
environment”.                                                                    neering principles, the most common answer was related to
   What do you do for sustainability in daily work life?                         improving the overall readability of the code through a variety
In contrast to how participants viewed sustainability in their                   of best practice activities i.e. code structure, use of comments
private life, there was a strong emphasis by seven participants                  etc. However, Mark and Ryan emphasized that this was pri-
on [technical] sustainability specifically related to code main-                 marily to achieve the “maintainability” of the code base. Mark
tainability and evolution associated with actions to ensure code                 explained that “I code in a maintainable way. I always code
readability, code efficiency, coding standards, as well as the                   so if I wasn’t here or available, other people would continue
importance of refactoring legacy code and code reusability.                      from me easy, without causing [memory] leaks”. This suggests
Mark highlighted the issues related to unsustainable practice                    that participants consider that maintainability is a core activity
stating that “the current project I am recoding is full of                       in ensuring that the code is [technically] sustainable. However,
memory leaks because it wasn’t coded well”. Ryan stated                          three participants (Claire, Ben and Steven) could not suggest
that “the team make sure the code is sustainable so others                       any software engineering principles specifically related to
can read it later on, change it and understand it”, which                        sustainability.
suggested that this was an important task for the entire team.
                                                                                    What software engineering practices do you consider
This was a view shared by the majority of participants. Claire
                                                                                 when building software systems? With regard to software
also highlighted the importance of the role of project man-
                                                                                 engineering practices, the most common answer was related to
agement in underpinning the process. Notably, the activities
                                                                                 software development process with four participants (Steven,
related to environmental sustainability previously mentioned
                                                                                 Claire, Thomas and Scott) stating that they employed SCRUM
for the context of private life (e.g., switching lights off, saving
                                                                                 and Agile development processes’ and practice. Claire stated
energy, and recycling) did not explicitly cross over into the
                                                                                 that everything starts off with “an ‘amigo’ meeting in which a
interviewees’ daily working practice.
                                                                                 small team discusses the software systems features, with tasks
B. Software Engineering Principles and Practice                                  be delegated into sprints”. Steven added that in addition to
                                                                                 the core SCRUM approach, they regularly employ techniques
   The second set of questions investigated the subjects’                        such as prototyping, code reviews, pair programming, and
knowledge with respect to software engineering principles and                    automated testing and deployment. The majority of responses
practices.                                                                       here are related to software engineering process.
   What are the main software engineering principles you
consider when working on a project?                                                What software engineering practices do you consider when
                                                                                 building software systems?
   With respect to software engineering principles, the most
common answer was related to improving “code readbility”                            When asked to relate sustainability with software engineer-
including coding style, formatting, naming, comments, coding                     ing practices, half of the participants (James, Scott, Mark,
conventions, and documentation. James emphasized that it                         Steven) fell back to the concepts of efficiency and maintain-
was critical that “others within the team must be able to                        ability. The remaining four participants (Ben, Ryan, Claire,
understand the code with minimal explanation”. In addition,                      Thomas) could not relate any of their current software en-
Thomas also mentioned the need to have comprehensive                             gineering practice to sustainability. Thomas stated, “I really
testing and utilise requirements traceability. The most in-                      don’t know if I contemplate sustainability at all when consid-
depth answer was given by Steven, who referred to the ability                    ering practice”.
to “adapt to change” and the ”agile principles”. He also                            As a general observation of the interview, even though all
claimed that “everything I apply within the project is to                        participants studied computing related subjects at a UK insti-
ensure sustainability, everything is considered for the long-                    tution of Higher Education with a common core of software
term effects of the software”. Two participants (Ben, Claire)                    engineering courses, the majority of the them struggled to
were unsure of what software engineering principles were.                        answer the questions related to the basic concepts of software
Ben stated that “as a Game Designer, I have never really                         engineering principles and practices, suggesting that these
considered software engineering principles”. However, the                        were not well understood by our volunteers. For instance, there
responses highlight a general misperception between software                     is a general lack of clarity on the difference between software
engineering principles7 such as GRASP8 , SOLID9 , YAGNI10 ,                      engineering principles and practice, suggesting that this dif-
                                                                                 ferentiation may not have been covered in the Huddersfield’s
   7 principle, (n). a primary assumption forming the basis of a chain of
                                                                                 curriculum.
reasoning [13]
   8 GRASP: General Responsibility Assignment Software Patterns
   9 SOLID: Single responsibility, Open-closed, Liskov substitution, Interface
segregation and Dependency inversion                                               11 DRY: Don’t Repeat Yourself
   10 YAGNI: You Ain’t Gonna Need It! [52]                                         12 KISS: Keep It Simple, Stupid!
C. Sustainability Requirements                                     that is sustainable, rather than having requirements that just
                                                                   meets functionality demands”. One interviewee was unable to
   The last set of questions referred to the participants’         give an answer.
understanding and ideas about requirements in general and             Elicitation, modelling, and evaluation of “sustainability
sustainability requirements in particular.                         requirements”. The last three questions of the interview
   What is your understanding of the term Functional               focused on elicitation, modelling and the evaluation of sus-
Requirement (FR)? Surprisingly, participants’ formal under-        tainability requirements13 .
standing of the term functional requirement deviated signif-          However, while the questions generated a small amount
icantly from how it is generally understood within the field       of discussion, participants either lacked confidence or were
of software and requirements engineering, defined in standard      unable to answer these questions with the majority opting for
software engineering textbooks [53] or SWEBOK [54]. Four           a fall back position of “I don’t think I could give you an
participants (Ryan, Scott, Claire, Steven) gave reasonable         answer to that”. Five participants (Ryan, Jamie, Scott, Mark,
definitions or examples of a functional requirement, such as       Steven and Thomas) were unsure how to formally analyze
“it is what the users of the product need to be able to achieve    requirements per se, which may in some instance be related
[...] ultimately the users have some action that they need to      to their working environment where “the specifications and
achieve [...] I guess it could be related to the controls of the   requirements are already drawn up for us” suggesting that
game. [...] a feature you expect to do something”. However,        this task is the responsibility of another team member. Most
four participants (Ben, James, Mark, Thomas) had no or             telling was that the majority of the participants were unable
an incorrect understanding of the term. For example, Mark          to relate any existing software and requirements engineering
stated, “no idea, I’d just assume it’s coding in a clean and       practice to the elicitation, modelling or the evaluation of sus-
clear way and following common practices”. These results are       tainability requirements. However, with regards to modelling
unexpected, as all participants work in the software industry.     a sustainability requirement, Thomas suggested , “I would
   What is your understanding of the term Non- Functional          use basic UML modelling so I could visualize the problem
Requirement (NFR)?                                                 or requirement...I don’t really have any examples”. Claire
   Similarly, participants’ understanding of the term non-         suggested employing “user stories then breaking that down
functional requirement also deviated significantly from the        into scenarios”; she believed that the method as a whole could
common view in the software/requirements engineering com-          be used to enable sustainability within software development.
munities. Three participants (James, Scott, Steven) suggested
that non-functional requirements were related to “perfor-                                     V. D ISCUSSION
mance” or “reliability”. However, four participants (Ryan,         A. Sustainability
Ben, Mark, Thomas, Claire) had no or an incorrect interpre-           All interviewees had a very broad understanding of sus-
tation of this term. Ben described it as “something that is        tainability, which was not limited to an environmental or
not compulsory, such as something that the project doesn’t         technical perspective. Only when prompted on the dimensions,
need to succeed, but something the user wants”, ignoring that      the interviewees related specific aspects, and, unsurprisingly,
fact that software is unlikely to be successfully used if non-     they could relate better to the environmental, economic, and
functional requirements are overlooked. Again, these results       technical dimensions than to the individual and social ones.
are somewhat surprising given respondents’ previous work in        Emphasis on action for sustainability in their private lives cen-
the software industry.                                             tered around the environmental dimension, including activities
   What is your understanding of the term “sustainability          like saving energy and recycling. In their work lives, it was
requirements”? When asked about “sustainability require-           focused strongly on technical sustainability, especially in terms
ments”, interviewees gave answers which were consistent            of maintainability, coding practices and team management.
with their understanding of sustainability. Four participants      These findings confirm the results reported for experienced
(Mark, Scott, James, Ben) considered the notions of ”time”,        software [32] and requirements [29] engineering practitioners,
”longevity” or ”maintenance”. For example, Mark suggested          and professional game developers [31]. Even though we cannot
that it is related to “making a system that will keep going        [yet] draw wider conclusions on the generalisability of the
over a long period of time [...], followed by examples of          results, this may serve as indicator that we need to better
coding standards (e.g. classes with less than 1000 lines,          infuse the different dimensions of sustainability into software
methods limited to 20 lines and a maximum of five variables        engineering education, and specifically give examples for the
within a method). Ryan suggested a relationship with reuse         individual and social dimensions [55].
when describing a sustainability requirement: “someone could         13 The questions are:
review my code and easily reword/restructure it as well as
                                                                     •   How would you elicit a sustainability requirement? Could you utilise
read it and understand it easily”. In general, explanations              any existing techniques?
lacked precision and showed confusion over the concept. For          •   How would you model a sustainability requirement? Could you utilise
example, James suggested that a sustainability requirement is            any existing techniques?
                                                                     •   How would you test or evaluate a sustainability requirement? Could
“a requirement that is sustainable over a prolonged period”.             you utilise any existing metrics and measures?
Thomas said that it was about “having a piece of software
B. Sustainability Requirements                                       This shows that there is a lack of coverage of sustainability
   The interviewees in this study sample appear to have no           in software engineering education, although there exist several
considered opinion on or grounded knowledge of sustainability        examples that propose principles and practices for including
in software engineering in general and sustainability require-       and explicitly addressing sustainability in SE; some of this
ments in particular. Though, when pressed for an answer, they        work is described in the background section of this paper. For
relate the notion of sustainability requirements to software         example, the principles for sustainabiltiy design are explicitly
maintainability and longevity, which is consistent with findings     listed in the Karlskrona Manifesto [10], and many practices
from [29]. Moreover, our interviewees were not clear about           are proposed by Naumann et. al. [15], Mahaux [59], Penzen-
the notions of functional and non-functional requirements            stadler [60], Betz [39], Chitchyan et. al., [58]. This is even
either. While the missing knowledge on sustainability could          more evident as the study has been conducted with recently
be attributed to lack of this topic’s coverage in Software           graduated students.
Engineering Curriculum, we are certain that the notion of               Overall, this leads to the question as to what education
functional and non-functional requirements is presented to the       about sustainable software engineering should look like and
students within several modules throughout their SE studies.         how to integrate sustainability principles and practice into
Yet, as per this study, this knowledge has not been retained,        it? Should there be a dedicated teaching module for this, or
suggesting that it has not been applied in common practice.          should the topic be spread across the regular SE modules?
   Furthermore, the SE community is well aware that non-             One way of integrating sustainability education into the SE
functional requirements (such as maintainability, extendabil-        curriculum is through infusing the sustainability considerations
ity, usability, security, etc.) are often the key factors in the     into SE courses as an overall aspect of long-term (strategic)
software systems’ success or failure. Yet, our interviewees          development, which prevents accumulation of sustainability
lack proper understanding of these. One explanation of this          debt (including technical debt, social and personal dissatisfac-
contradiction may lie in the “early career” stage of our             tion, economic losses, and environmental costs) [39]. Indeed,
sample. As novice developers, they are likely to be focused          sustainability is an ever present concern in SE, even if not
on the specific functionality implementation allocated to them,      always explicitly acknowledged [10].
while senior colleagues, such as team leads, or software                          VI. C ONCLUSIONS AND O UTLOOK
architects handle the concerns of non-functional (including
sustainability-related) requirements. This view is supported by         This paper reported results of an interview study on the
prior work (e.g., [56]) where senior engineers demonstrate key       knowledge of novice software developers with respect to
engagement with NFRs.                                                sustainability, sustainability requirements, and the relationship
   Consideration of the concept of a sustainability require-         of these concepts to software engineering principles and prac-
ments did not lead our interviewees to relate it to their            tices.
practice (e.g., one even interpreting environmental dimension           Our findings highlight that sustainability is generally un-
of sustainability as having to do with the weather). While dis-      derstood in its broadest sense i.e. ‘capable of being endured’
appointing, this result is not entirely unexpected, as the notion    but it also often encompasses environmental, economic, and
of sustainability is so very broad that, at first encounter, it      technical dimensions. However, we also observe a gap in
seems not connected to a specific task. Moreover, requirements       understanding the notions of social and individual sustainabil-
that directly relate to sustainability are not perceived as such,    ity, which is corroborated by the findings from other similar
as they are traditionally treated under segregated headings          studies [33], [32], [31].
(e.g., accessibility and usability directly relate to the individ-      The results also suggest that while sustainability is not a
ual sustainability dimension; maintainability to the technical       primary or overarching focus of our interview participants’
and economic dimension, etc.). To address this challenge,            cohort, the concept of sustainability is valued highly by
related research on collecting domain-specific sustainability        these novice software developers, with a strong emphasis
requirements has recently been initiated [57], as well as an         on attaining technical sustainability through maintainability
effort to illustrate sustainability relevance to software process    related activities. In this context, the drive towards achieving
and products with examples [12], [58]. A collection of such          maintainability aligns with ‘the ease with which a software
practical examples could serve as a good educational resource        system or component can be modified to correct faults, im-
for the novice practitioners, such as our interviewees.              prove performance or other attributes, or adapt to a changed
                                                                     environment’ [54]. As such, it is important to recognize that
C. Software Engineering: Principles and Practice                     sustainability is an explicit consideration even if the primary
   From our interviews we observe, that none of the partici-         focus of the system under design is not sustainability; a view
pants considered principles specifically dedicated to sustain-       supported by the Karlskrona Manifesto [10].
ability. Some participants have their own sets of practices             In addition, the results highlight a deficit in the knowl-
that pertain to system longevity and maintainability, and also       edge of basic software engineering theory and practice as
used agile methods (like code reviews and pair programming).         it is generally understood by the software and requirements
However, four out of eight interviewees, were not able to            engineering community as a whole, as well as a misalignment
relate any software engineering practice with sustainability.        with existing software engineering principles and practice in
relation to sustainability. While these results by themselves                   [8] Z. Merali, “Computational science: Error, why scientific programming
provide a Fisheye view lens of software engineering and                             does not compute,” Nature, vol. 467, no. 7317, pp. 775–777, 2010.
                                                                                [9] M. M. Lehman, “Software’s future: Managing evolution,” IEEE soft-
sustainability, it raises serious questions regarding whether                       ware, vol. 15, no. 1, pp. 40–44, 1998.
software engineering education at this particular institution                  [10] C. Becker, R. Chitchyan, L. Duboc, S. Easterbrook, B. Penzenstadler,
is fit for purpose or whether it is symptomatic of a larger                         N. Seyff, and C. C. Venters, “Sustainability design and software:
                                                                                    The karlskrona manifesto,” in Proceedings of the 37th International
problem in the software engineering curriculum. To address                          Conference on Software Engineering - Volume 2, ser. ICSE-SEIS ’15,
this question, we require further research and evidence.                            May 2015, pp. 467–476.
   We note that the education sector as a whole has an                         [11] C. C. Venters, N. Seyff, C. Becker, S. Betz, R. Chitchyan, L. Duboc,
                                                                                    D. McIntyre, and B. Penzenstadler, “Characterising sustainability re-
important role to play in ensuring that software designers of                       quirements: A new species, red herring, or just an odd fish?” in Pro-
the future fully understand the concept of sustainability and                       ceedings of the 39th International Conference on Software Engineering:
its integral relationship to the field of software engineering by                   Software Engineering in Society Track, ser. ICSE-SEIS ’17, 2017, pp.
                                                                                    3–12.
bridging the chasm between software engineering and sustain-
                                                                               [12] C. Becker, S. Betz, R. Chitchyan, L. Duboc, S. M. Easterbrook,
ability. Education presents a key avenue for improvement in                         B. Penzenstadler, N. Seyff, and C. C. Venters, “Requirements: The Key
understanding sustainability. Educators need to consider how                        to Sustainability,” IEEE Software, vol. 33, no. 1, pp. 56–65, Jan-Feb
to integrate sustainability into software engineering curricula                     2016.
                                                                               [13] “Oxford English Dictionary Online, 2nd edition,” http://www.oed.com/,
and articulate the competencies required for successful sus-                        July 2003.
tainability design. In this context, there is a need to expand                 [14] G. H. Brundtland and UN World Commission on Environment and
the pool of examples of how to include sustainability in                            Development, Our common future. Oxford University Press, 1987.
                                                                               [15] S. Naumann et al., “The greensoft model: A reference model for green
software engineering curricula, particularly from the inter-                        and sustainable software and its engineering,” Sustainable Computing:
national organizations that provide curricula guidelines. In                        Informatics and Systems, pp. 294–304, 2011.
ACM/IEEE SWE Curricula, sustainability is mentioned only                       [16] H. Koziolek, “Sustainability evaluation of software architectures: a
                                                                                    systematic review,” in ACM SIGSOFT Conf. QoSA and ISARCS. ACM,
once. Similarly, in the SWEBOK curricula, sustainability is                         2011, pp. 3–12.
mentioned twice in the software economics area.                                [17] A. Molla, V. A. Cooper, and S. Pittayachawan, “It and eco-sustainability:
   This paper provides further evidence that we must consider                       Developing and validating a green it readiness model,” ICIS 2009
                                                                                    Proceedings, p. 141, 2009.
the question of what would sustainable software engineering
                                                                               [18] B. Penzenstadler, “Towards a definition of sustainability in and for soft-
education would look like? To further investigate the current                       ware engineering,” in Proceedings of the 28th Annual ACM Symposium
situation with regards to software engineering education and                        on Applied Computing, ser. SAC ’13, 2013, pp. 1183–1185.
sustainability, future work will extend this study by conducting               [19] L. M. Hilty and B. Aebischer, “Ict for sustainability: An emerging
                                                                                    research field,” in ICT Innovations for Sustainability. Springer, 2015,
more studies in different organizations from different coun-                        pp. 3–36.
tries. Based on the results of theses studies, we are planning                 [20] R. C. Seacord et al., “Measuring software sustainability,” in ICSM, Sept
to work on curricula guidelines and recommendations.                                2003, pp. 450–459.
                                                                               [21] C. Calero et al., “Quality in use and software greenability,” in Workshop
                   ACKNOWLEDGEMENTS                                                 RE4SuSy, 2014.
   We would like to express our sincere thanks and gratitude                   [22] C. C. Venters, C. Jay, L. Lau, M. K. Griffiths, V. Holmes, R. Ward,
                                                                                    J. Austin, C. E. Dibsdale, and J. Xu, “Software sustainability: The mod-
to our friend and colleague Professor Christoph Becker14 ,                          ern tower of babel,” in Proceedings of the Third International Workshop
University of Toronto (Canada), for his significant contribu-                       on Requirements Engineering for Sustainable Systems (RE4SuSy 2014),
tions and insights into the ongoing research agenda of the                          2014.
                                                                               [23] C. Calero and M. Piattini, Green in software engineering. Springer,
Karlskrona Consortium into sustainability design15 . We also                        2015.
gratefully acknowledge the eight participants who willingly                    [24] C. C. Venters, L. Lau, M. K. Griffiths, V. Holmes, R. R. Ward, C. Jay,
gave up their time to participate in this study.                                    C. E. Dibsdale, and J. Xu, “The blind men and the elephant: Towards
                                                                                    an empirical evaluation framework for software sustainability,” Journal
                               R EFERENCES                                          of Open Research Software, vol. 2, no. 1, 2014.
                                                                               [25] B. Penzenstadler, A. Raturi, D. Richardson, and B. Tomlinson, “Safety,
 [1] H. Walkington, “Students as researchers: Supporting undergraduate              Security, Now Sustainability: The Nonfunctional Requirement for the
     research in the disciplines in higher education,” 2015.                        21st Century,” IEEE Software, vol. 31, no. 3, pp. 40–47, May 2014.
 [2] H. Pham, Software reliability. Wiley Online Library, 1999.                [26] J. L. Ramsey, “On not defining sustainability,” Journal of Agricultural
 [3] F. P. Deek, J. A. M. McHugh, and O. M. Eljabiri, Strategic Software            and Environmental Ethics, vol. 28, no. 6, pp. 1075–1087, 2015.
     Engineering: An Interdisciplinary Approach. Auerbach Publications,        [27] J. A. Tainter, “Social complexity and sustainability,” Journal of Ecolog-
     2005.                                                                          ical Complexity, no. 3, pp. 91–103, 2006.
 [4] R. Kitchin and M. Dodge, Code/space: Software and everyday life. Mit      [28] B. Knowles et al., “Exploring sustainability research in computing:
     Press, 2011.                                                                   Where we are and where we go next,” in UbiComp. ACM, 2013,
 [5] A. Geist and R. Lucas, “Major computer science challenges at exascale,”        pp. 305–314.
     Journal of High Performance Computing Applications, vol. 23, no. 4,       [29] R. Chitchyan, C. Becker, S. Betz, L. Duboc, B. Penzenstadler, N. Seyff,
     pp. 427–436, 2009.                                                             and C. C. Venters, “Sustainability design in requirements engineering:
 [6] T. Hey, S. Tansley, K. M. Tolle et al., The fourth paradigm: data-             State of practice,” in Proceedings of the 38th International Conference
     intensive scientific discovery.    Microsoft research Redmond, 2009,           on Software Engineering Companion, ser. ICSE-SEIS ’16, 2016, pp.
     vol. 1.                                                                        533–542.
 [7] C. Goble, “Better software, better research,” IEEE Internet Computing,    [30] I. Manotas, C. Bird, R. Zhang, D. Shepherd, C. Jaspan, C. Sadowski,
     vol. 18, no. 5, pp. 4–8, Sept 2014.                                            L. Pollock, and J. Clause, “An empirical study of practitioners’ per-
                                                                                    spectives on green software engineering,” in Proceedings of the 38th
  14 Christoph Becker: https://ischool.utoronto.ca/christoph-becker/
                                                                                    International Conference on Software Engineering, ser. ICSE ’16, 2016,
  15 Sustainability Design: http://sustainabilitydesign.org/                        pp. 237–248.
[31] J. Kasurinen, M. Palacin-Silva, and E. Vanhala, “What concerns game          [57] M. Al Hinai and R. Chitchyan, “Engineering requirements for social
     developers?: A study on game development processes, sustainability and            sustainability,” ICT4S, 2016.
     metrics,” in Proceedings of the 8th Workshop on Emerging Trends in           [58] R. Chitchyan, W. Cazzola, and A. Rashid, “Engineering sustainability
     Software Metrics, ser. WETSoM ’17, 2017, pp. 15–21.                               through language,” in Proceedings of the 37th International Conference
[32] I. Groher and R. Weinreich, “An interview study on sustainability                 on Software Engineering - Volume 2, ser. ICSE ’15, vol. 2. IEEE, 2015,
     concerns in software development projects,” in Proceedings of the                 pp. 501–504.
     43rd Euromicro Conference on Software Engineering and Advanced               [59] M. Mahaux, P. Heymans, and G. Saval, “Discovering sustainability
     Applications, ser. SEAA ’17, 2017.                                                requirements: an experience report,” in International Working Confer-
[33] R. Chitchyan, I. Groher, and J. Noppen, “Uncovering sustainability                ence on Requirements Engineering: Foundation for Software Quality.
     concerns in software product lines,” Journal of Software: Evolution and           Springer, 2011, pp. 19–33.
     Process, vol. 29, no. 2, February 2017.                                      [60] B. Penzenstadler, “Infusing green: Requirements engineering for green
[34] M. Mahaux, P. Heymans, and G. Saval, “Discovering sustainability                  in and through software systems.” in RE4SuSy@ RE, 2014, pp. 44–53.
     requirements: an experience report,” in Intl. Working Conf. REFSQ,
     2011, pp. 19–33.
[35] K. Roher and D. Richardson, “A proposed recommender system for
     eliciting software sustainability requirements,” in Workshop USER, 2013,
     pp. 16–19.
[36] ——, “Sustainability requirement patterns,” in Requirements Patterns
     (RePa), 2013 IEEE Third Intl Workshop on, 2013, pp. 8–11.
[37] J. Cabot et al., “Integrating sustainability in decision-making processes:
     A modelling strategy,” in 31st ICSE. IEEE, 2009, pp. 207–210.
[38] S. A. Kocak, “Green software development and design for environmental
     sustainability,” in 11th International Doctoral Symposium an Empirical
     Software Engineering (IDOESE 2013). Baltimore, Maryland, vol. 9,
     2013.
[39] S. Betz, “Sustainability aware process management using xmlnets,” in
     Proceeding of the 28th EnviroInfo Conference, 2014.
[40] C. K. Makropoulos, K. Natsis, S. Liu, K. Mittas, and D. Butler,
     “Decision support for sustainable option selection in integrated urban
     water management,” Environ. Model. Softw., vol. 23, no. 12, pp. 1448–
     1460, Dec. 2008.
[41] S. A. Kocak, G. I. Alptekin, and A. B. Bener, “Integrating environmental
     sustainability in software product quality,” in RE4SuSy 2015: Require-
     ments Engineering for Sustainable Systems, vol. 1416. CEUR, 2015,
     pp. 17–24.
[42] “ISO/IEC 25010:2011: Systems and software engineering. Systems and
     software quality requirements and evaluation (SQuaRE),” 2011.
[43] M. Z. Huber, L. M. Hilty, and M. Glinz, “Uncovering sustainability
     requirements: An exploratory case study in canteens,” in RE4SuSy 2015:
     Requirements Engineering for Sustainable Systems, vol. 1416. CEUR,
     2015, pp. 35–44.
[44] M. Glinz, “Rethinking the notion of non-functional requirements,” in
     Third World Congress for Software Quality, Sep. 2005, pp. 55–64.
[45] J. Eckhardt, A. Vogelsang, and D. M. Fernández, “Are ”non-functional”
     requirements really non-functional?: An investigation of non-functional
     requirements in practice,” in Proceedings of the 38th International
     Conference on Software Engineering, ser. ICSE ’16, 2016, pp. 832–842.
[46] E. Fossey, C. Harvey, F. McDermott, and L. Davidson, “Understanding
     and evaluating qualitative research*,” Australian and New Zealand
     Journal of Psychiatry, vol. 36, no. 6, pp. 717–732, 2002.
[47] J. W. Creswell, Qualitative inquiry & research design: choosing among
     five approaches, 3rd ed. SAGE, 2013.
[48] P. Mayring, “Qualitative Content Analysis,” in Forum Qualitative Sozial-
     forschung/Forum: Qualitative Social Research, vol. 1, no. 2, 2000.
[49] Simple Collaborative Qualitative Analysis. [Online]. Available: http:
     //www.saturateapp.com
[50] “Fisheye lens view: Interview guide & codebook,” http://tinyurl.com/
     h8dz4s3.
[51] P. A. Ochieng, “An analysis of the strengths and limitation of qualitative
     and quantitative research paradigms,” Problems of Education in the 21st
     Century, vol. 13, June/2009 2009.
[52] R. E. Jeffries, A. Anderson, and C. Hendrickson, Extreme Programming
     Installed. Addison-Wesley Longman Publishing Co., Inc., 2000.
[53] I. Sommerville, Software engineering, tenth, global ed.           Pearson
     Education Limited, 2016.
[54] P. Bourque, R. E. Fairley, and I. C. Society, Guide to the software
     engineering body of knowledge: SWEBOK, version 3.0. IEEE, 2014.
[55] R. Chitchyan, S. Betz, L. Duboc, B. Penzenstadler, S. Easterbrook,
     C. Ponsard, and C. Venters, “Evidencing sustainability design through
     examples,” in Fourth International Workshop on Requirements Engineer-
     ing for Sustainable Systems (RE4SuSy), August 2015.
[56] D. Ameller, C. P. Ayala, J. Cabot, and X. Franch, “Non-functional
     requirements in architectural decision making,” IEEE Software, vol. 30,
     no. 2, pp. 61–67, 2013.