<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>Validation Study of a Framework for Sustainable Software System Design and Development</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Shola Oyedeji</string-name>
          <email>shola.oyedeji@lut.fi</email>
          <xref ref-type="aff" rid="aff0">0</xref>
          <xref ref-type="aff" rid="aff1">1</xref>
          <xref ref-type="aff" rid="aff2">2</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Mikhail .O. Adisa</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
          <xref ref-type="aff" rid="aff1">1</xref>
          <xref ref-type="aff" rid="aff2">2</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Annika Wolf</string-name>
          <email>annika.Wolff@lut.fi</email>
          <xref ref-type="aff" rid="aff0">0</xref>
          <xref ref-type="aff" rid="aff1">1</xref>
          <xref ref-type="aff" rid="aff2">2</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>Department of Computer Engineering and Computer Science, California State University Long Beach Long Beach, USA LUT School of Engineering (LENS) LUT University Lappeenranta</institution>
          ,
          <country country="FI">Finland</country>
        </aff>
        <aff id="aff1">
          <label>1</label>
          <institution>IT Service Management Consultant IT Solutions Abuja</institution>
          ,
          <country country="NG">Nigeria</country>
        </aff>
        <aff id="aff2">
          <label>2</label>
          <institution>LUT School of Engineering Science (LENS) LUT University Lappeenranta</institution>
          ,
          <country country="FI">Finland</country>
        </aff>
      </contrib-group>
      <abstract>
        <p>-Sustainability in software design is an evolving area that requires more practical guidance on how software engineers and businesses could innovate and design software systems that consider sustainability as a guiding principle for supporting a sustainable environment, reducing the negative impact of ICT and at the same time promoting software system design for sustainability. This paper presents our early results for validating a Framework for Sustainability of Software System Design (FSSSD) based on the Software Sustainability Design Catalogue (SSDC). The SSDC exemplifies the use of Karlskrona Manifesto principles for sustainability design and how to promote sustainability design principles for software systems. Index Terms-Sustainable design, sustainability, software sustainability, information and communication technology, Karlskrona manifesto, Sustainability design principles</p>
      </abstract>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>I. INTRODUCTION</title>
      <p>
        Sustainability is receiving a wide range of research from
different sectors. Currently, there is not enough research results
with guidelines and frameworks to support software designers
and companies on how to design and develop software with
sustainability at the core [1]. One of the main problems for
sustainability in software design is that for software designers
there are few existing tools that wrap core principles of
sustainability together which can support effective software
sustainability design and development [
        <xref ref-type="bibr" rid="ref14">2</xref>
        ]. For companies, the
challenge is that there is little understanding of how sustainability
can be understood by software and requirements engineering
professionals to facilitate sustainability design as an established
part of the software development process within companies
[3][4][5].
      </p>
      <p>The sustainable development goals (SDGs) [6] in 2015 got
signed by more than 190 world leaders, this shows the
importance of sustainability today in all aspects of our lives.</p>
      <p>Birgit Penzenstadler
Though there is no direct mention of software sustainability in
the 17 SDGs, software as a catalyst for all sectors of the
economy [7] serves as a key element for the implementation and
actualization of those SDGs. According to the 2016 mobile
industry impact report [8], the United Nations Sustainable
Development Goals provide the opportunity for engagement to
address the most pressing global challenges, but they cannot be
realized without the business community. The report stresses
the need for companies to implement the SDGs, working with
governments and the international community to expand
connectivity, lower barriers to access, and build a future of dignity
and opportunity, where no one is left behind and ensure that
tools and applications are developed with vulnerable
communities in mind [8].</p>
      <p>
        Sustainable development is also driving software
innovations for creating new opportunities of cutting costs, adding
value and for gaining competitive advantage [9]. García-Berna
et al. [10] points out the practices applied by practitioners in
companies for sustainability and the need for standards as a
way of seeking more sustainable software businesses. The
importance of sustainability as a driving force for companies is
further highlighted in these reports: Sustainability Nears a
Tipping Point [
        <xref ref-type="bibr" rid="ref12">11</xref>
        ]; Ericsson energy and carbon report [12];
Microsoft 2015 Citizenship Report [13]. In summary, software is
a core of all human activities today and a major facilitator in
the way humans produce and use products and services [14].
The way software is designed and the requirements to ensure
sustainability in software design are factors that are challenging
for software designers, requirement engineers and companies
[15].
      </p>
      <p>The Karlskrona Manifesto for Sustainability Design
(KMSD) [16] was initiated as a starting point for tackling this
challenges in software engineering. Based on these KMSD
principles and the Software Sustainability Design Catalogue
(SSDC) [1], the Framework for Sustainability of Software
System Design (FSSSD) was created [1]. This paper presents the
first results of applying the Framework for Sustainability of
Software System Design (FSSSD) [1].</p>
      <p>The next section covers related research work. Section III
presents the study design. Section IV covers the first case study
and section V details the second case study. Discussion is in
section VI and concluding remarks in section VII.</p>
    </sec>
    <sec id="sec-2">
      <title>II. BACKGROUND</title>
      <p>Software development practices and processes that are
widely used in industry for software design and development
lack in addressing sustainability [17]. There is currently no
single point of reference for researchers and practitioners where
the sustainability measures are gathered and exemplified [26].
The issue of lack of understanding on how to effectively and
efficiently integrate the different sustainability dimensions
(economic, social, individual, environmental and technical)
[18] into software design, development and wider engineering
processes [9] [19] has hindered the adoption of sustainability in
software development.</p>
      <p>
        There have been different research efforts suggesting the
need to further research on how sustainability can be supported
in software requirements and design stages for all the different
sustainability dimensions [20] [21] [22]. Further research also
shows sustainability requires multidimensional and
interdisciplinary approach [3][7][23][
        <xref ref-type="bibr" rid="ref28">24</xref>
        ][25] in order to fully achieve
sustainability in software design, development and
measurement.
      </p>
      <p>From the requirements engineering phase, sustainability has
been considered as a non-functional requirement [26][27][28],
and Roher et al. [29] suggests the use of sustainability
requirement patterns (SRPs) as a way to guide software requirements
engineers in eliciting sustainability requirements in the
requirements engineering process. However, there is a lack of
examples to show how these are applied in the industry.</p>
      <p>Researchers from the Human Computer Interaction (HCI)
community believe sustainable HCI can facilitate and support
sustainability in the design and development of new interfaces
to promote sustainability awareness [30]. Froehlich et al. [31]
show eco feedback can serve as a key way of promoting
sustainability awareness among users of software systems. One
key example of an eco-feedback application [32] shows a
positive result in persuading and changing users habit towards
sustainability. Successful application of eco feedback is when
information has been tailored to encourage users towards
sustainability through user emotional engagement [33] [34].</p>
      <p>Some of the design issues in design of sustainability for
better user experience of software systems are highlighted by
Kem-Laurin [35]. Kem-Laurin propose the use of sustainability
user experience framework as a way to guide designers to
mitigate these problems. The challenge according to Eli Blevis [36]
and Fallman [37] is that sustainability is not yet a core part of
HCI. This has hindered the ability of designers to properly
evaluate design choices for software systems especially with
the different sustainability dimensions.</p>
      <p>The challenges covered in this background section motivate
the application of FSSSD to two case studies in order to show
and suggest how to better support sustainability in software
design and development.</p>
      <p>III. STUDY DESIGN FOR FRAMEWORK VALIDATION</p>
      <p>This section describes the Framework for Sustainability of
Software System Design (FSSSD) and the rationale behind
choosing the two case studies used in the research.</p>
      <p>The FSSSD (Figure 1) was created to assist developers to
incorporate sustainability goals and requirements during
software system design and development covering the software
development life-cycle (SDLC) phases. For the purpose of
better understanding, the FSSSD (Figure 1) is transformed into
tabular form (Table 1) [1].</p>
      <p>The approach applied in the selection of each case study
was to choose two different case studies where one case study
has the ultimate goal of sustainability from the beginning and
the other case study uses the framework to improve an existing
system.</p>
      <p>The goal is to see what difference will occur from these two
different case studies in different application context. The first
case study - about a pension benefit tracker application - does
not have sustainability as the central core and the second case
study - about an energy usage display for university staff and
students - is motivated by sustainability.</p>
      <p>IV. CASE STUDY ONE: PENSION BENEFIT TRACKER</p>
      <p>APPLICATION
The pension benefit tracker is an application from a pension
company in Nigeria that wants to track pension benefit
applications submitted by clients from all over the company’s
branches in different states of Nigeria. Currently, the pension
applications are done manually from each branch and those
applications are sent via courier service to the head office. This usually
causes the following problems:
1. Zonal managers don’t have direct access to know the
status of applications submitted through them and
have to directly place phone calls to the Head office to
know the application status.
2. Customer service staff are unable to know why an
application is pending, unless they contact the benefit
department.
3. Time consumption, as all status updates are through
customer service at the head office alone.
4. Files can go missing in transit because application
files are handled manually.
5. Double application and too much physical
involvement because of follow up in person</p>
      <p>The company intended to develop a new pension benefit
application tracker application for these key stakeholders, the
benefit department, the customer service unit, the zonal
managers and the clients with the aim of:
1. Identifying ways of improving the pension benefit
application process and enhance communication.
2. Designing and implementing a web-based solution
that will ensure effective and efficient benefit
processing for users.</p>
      <p>The below Figure 2 is the first Use case diagram for the
application.</p>
      <p>Fig 2. Use Case diagram pension benefit tracker</p>
      <p>Figure 2 shows the use case diagram of the system for
pension benefit tracker application after initial analysis. Figure 3
presents the process model of the pension benefit application
after a second analysis, factoring in all the aforementioned
problems without using FSSSD. Figure 3 shows that
sustainability was not the core of this case study, based on the process
model, as stakeholders are just interested in solving the
problems stated in the case study.</p>
      <p>Table 2 presents the details for applying FSSSD to the pension
benefit tracker application (case study one). The documentation
for this case study using FSSSD covers the project initiation,
user requirements and system requirements phases only (see
Table 2) because that is the current development stage of the
project.</p>
      <p>Sustainability
Concepts, Methods and
Tools
Motivated by the
cradle to cradle
approach ensuring
that the pension
tracker application is
design and
developed in a way that it
can be reused for
future pension
related purposes and
easily integrated
with other bigger
pension system
within the company
Sustainability
requirement Template
Social and
individual dimension of
sustainability</p>
      <p>Indicators /Measure / Metric
1. How many state branches can
easily integrate the systems with
less Backlog Management Index
(BMI)?
2. What is the number of reports
from IT staff about how to
improve system energy efficiency?
3. How satisfied are the
developers with the development of
the application
How efficient is benefit
department able to track new pension
benefit applications and send
notification successfully
1. How satisfied are users with
visual problem with the
magnifying display?
2. Do users use the option of
email notification and does it
reduce company cost for sending
SMS?
3. How many positive responses
came from users base on the
“Save the planet, Reduce
environmental waste” tag message?
Fig 3. New Process Model for Pension Application after second analysis
Phase 1. Project Definition
Provide end users with easy to use interface for
tracking pension payment, ensure each module
for tracking can be updated to include new
branches,
Provide flexibility such as bulk and single
upload, ensure easy integration with other existing
pension systems, present report of system usage
to track energy consumption in a way to educate
users about sustainability, add bug reports
Phase 2. User Requirements Definition
1. Provide tracking of pension benefit payment
application from request submission to payment
2. Status notification should be sent to users after
each stage of the pension benefit application
Phase 3. System Requirements Definition
1. The pension tracker application should be
accessible online via web at any branch
2. The application should have ability to enable
Managers, pensioners and other stakeholders
check application status
3. Provide automatic status communication and
notification at each stage of benefit application
4. Allow bulk or single file upload
5. Provide SMS authorization from managers in
benefit department
6. Send SMS notification to applicants
7. Send Incomplete documentation notification to
Design for:
Easy integration,
Reusability,
Developers work satisfaction,
Maintainability,
Energy efficiency
Reduce development
increase efficiency</p>
      <p>cost,
Design for efficiency,
sustainability awareness
benefit department staff
8. Provide email notification as an option for all
users
9. Provide option of different display to magnify
fonts for users with visual problems
10. Provide option to preview pension
application and save electronically
11. Add a tag message below each notification
“Save the planet, Reduce environmental waste”
12. Provide energy report for system usage</p>
      <p>After application of the FSSSD with the sustainability
design catalogue (SSDC), see Table 2, the IT department made
some changes to the system requirements such as addition of
the following system requirements in Table 2, SDLC phase 3:
1. Email notification option instead of only SMS
function as seen in Figure 3 in which only SMS is shown
(system requirement 8 in Table 2).
2. Provide option of different display to magnify fonts
for users with visual problems especially older staff
(system requirement 9 in Table 2).
3. Provide option to preview pension application and
save electronically instead of printing and filling
locally to reduce cost, paper waste and energy usage
(system requirement 10 in Table 2)
4. Add a tag message below each notification “Save the
planet, and reduce environmental waste” to raise
sustainability awareness among staff and clients (system
requirement 11 in Table 2).
4. How many initiatives were
suggested from IT department
base on the system energy
report?
5.</p>
      <p>An energy report that enables developers to improve
efficiency (system requirement 12 in Table 2).</p>
      <p>V. CASE STUDY TWO: ENERGY USAGE AND CARBON EMISSION</p>
      <p>DISPLAY FOR UNIVERSITY STAFF AND STUDENTS</p>
      <p>This is a university setting project to raise the awareness of
the public (university staff and students) about energy usage
and the carbon emissions through activities in the university.
The project requires a web application interface which will
display the energy usage and carbon emission. The goal is to let
the public know more about the electricity consumption of each
building in the university and understand the relation between
the electricity consumption and carbon emission (CO2).
Using the FSSSD, the involved students and their supervisors
documented the project to show how sustainability was
considered in the project (see Table 3). Figure 4 shows the interface
design for the project and Figure 5 covers an overview of the
sustainability business canvas for the project.
2. The application should translate the
carbon emission data base on energy usage
into meaningful information for better user
understanding such as distance between
Lappeenranta and other cities
3. The web interface should allow users
participate in the weekly challenge
4. Users are able to share their weekly
challenge results via Facebook and
Twitter.
5. The application should allow users form
community of interest for different
sustainability goals.
6. Provide API to allow for easy
integration with other applications
Phase 4. Analysis and Design
1. Identify the first, second and third order
impact of the application on user energy
usage and sustainability awareness
2. Find areas to improve the application
implementation base on the different
sustainability dimensions especially
environment, social and technical dimensions
Phase 5. Development
Design for sustainability
awareness, reuse,
efficiency and localization
Design for sustainability
awareness, efficiency,
reuse, design for module
replicability, design for
easy service and
maintenance</p>
      <p>Sustainability analysis
radar chart was used for
the sustainability analysis
to show the he first, second
and third (immediate,
enabling, and structural)
impacts of the application.</p>
      <p>Cradle to cradle concept
influence the development
to develop each module in
the application in a way
that support evolution as
user requirements changes
over time and ensuring
sustainability is the core of
all development
weekly challenge?
3. Does the application to form
community of different
sustainability goals?
4. Can users successfully share
their weekly challenge on Facebook
and Twitter?
5. Does the API allow easy
information access?
1. What is the potential percentage
of energy usage reduction in the
university?
2. What is the level of user
awareness overtime about energy usage
and carbon emission?
3. What is the impact of the user
community for users’ motivation
towards sustainability within the
university?
1. What is the defect density of the
application?
2. What is the energy efficiency of
the application?
3. How many modules relating to
sustainability awareness was
successfully developed?
4. Can users successfully use the
application for all application
functions such as join a community,
participate and share weekly
sustainability results, understand
displayed energy usage and carbon
emission information?</p>
      <p>Fig 4.Sustainability awareness via energy usage interface</p>
      <p>Fig 5. Sustainable Business Canvas for Case Study Two [38]</p>
    </sec>
    <sec id="sec-3">
      <title>VI. DISCUSSION</title>
      <p>For the project initiation in the first case study, normally
project managers will only evaluate projects by considering
whether the software system meets all user requirements after
development and testing as a yardstick for satisfying all project
requirements. The application of FSSSD in case study one
(Table 2) shows that indicators used for evaluating the project up
to the current development stage included the level of
developer satisfaction (individual dimension of sustainability) and the
number of IT staff reporting on how to improve the system
energy efficiency (environment and technical dimension). This
confirms a new perspective towards software project evaluation
with sustainability dimensions now considered by stakeholders
in case study one. The use of FSSSD also led to new system
requirements (Table 2) with the potential to improve the system
efficiency and consideration of sustainability based on the
system context.</p>
      <p>Based on the initial response from stakeholders in case
study one, it indicates that as a company their major interest
was to check if FSSSD - as guide in the application of
sustainability in software system design and development - would
save them cost and improve staff productivity. The use of
developers satisfaction for the pension benefit tracker is one
example because the company believes if there is means of
checking staff satisfaction, it could offer a means of improving
working conditions which will in turn improve productivity
over time. This will help them reduce the cost of operations
and improve profit margin.</p>
      <p>Case study two provides a different use of FSSSD as
sustainability is the core of the application design. As noted in [33]
[34], with better tailored information through eco feedback,
user habits can change positively towards sustainability over
time. The second case study (see Table 3, Figures 4 and 5)
shows the presentation of energy usage data converted into
carbon emission. With the use of FSSSD as guide, the
application in case study two was designed in a way that the carbon
emission information was displayed in order to educate users
about their energy consumption habits in each department. The
system presented the percentage of carbon emission in form of
distance between one city to another with the goal to provide
better understanding for the public about the impact of their
energy consumption on the environment.</p>
      <p>Feedback and comments (Table 4) from stakeholders in
case study one and two indicates that developers and engineers
complained there are few industry case studies for software
development that shows how sustainability was applied. The
second challenge was in motivating software requirements
engineers and designers to incorporate the use of the new
sustainability artifacts for sustainability in requirements and software
development because most of them are used to the old ways of
developing software systems and therefore require extensive
discussion on the usage of the artifacts in FSSSD.</p>
      <p>In general, the early feedback and comments (Table 4) from
case study one and two shows that the Framework for
Sustainability of Software System Design (FSSSD) provides guidance
and support for sustainability in software design requirements
and development. The tools, methods and concepts provided as
Software
developer,
Project
coordinator
System
analyst, software
developer
System
analyst,
Programmers,
Software
developer</p>
      <p>User requirement
definition
System
Requirements Definition
Analysis and
Design
1. The SSDC was good way to understand the
different aspect of sustainability for different kind of
software system. The SSDC made it possible for me and
my team to know more about sustainability in
software development with those guidelines provided for
each software system.
2. The FSSSD provides new insight for sustainability
in software project with consideration of
sustainability principles
3. Combination of the SSDC and FSSSD provides an
avenue to consider our software impacts and see how
we can minimise it.
4. FSSSD introduces new methods for evaluating our
applications especially the environmental and
individual dimensions of sustainability
5. The Sustainable Business Canvas brings in a
totally new factors into software project definition with
sustainability concepts and dimensions as guide
1. The sustainability requirement template was
useful as guide during requirement gathering because it
provides us with means of discussing sustainability
with users and categorising user requirements base
on sustainability dimensions
1. I was able to learn new things about how
sustainability can influence gathering system requirements
and identifying new system requirements using the
FSSSD
2. The goal model diagram is really a good tool to
breakdown sustainability goals base on requirements
into business, usage and system goal.
3. The goal model diagram made it easy to explain,
discuss and improve the project goals and system
requirements using the business, usage and system
goal diagram.
1. The sustainability goals and suggested tools from
FSSSD was a good starting point to guide us during
the analysis and design phase.
2. The sustainability analysis radar chat was a new
interesting tool because it shows some new
requirements to add after brainstorming on each of the first,
second and third impacts
sample in the framework helped in providing new insights into
how sustainability can be incorporated into software project
design and development especially the Sustainable Business
Canvas, Goal model, Sustainability Requirement Template,
Biomimicry, Cradle to cradle concept and Sustainability
Analysis Radar Chat diagram. In addition, FSSSD also persuades
stakeholders to rethink their software project with sustainability
as a means of developing a better product that is cost effective
over a long time and supports good corporate social
responsibility. Table 4 summarizes the feedback on the usage of FSSSD
from the case studies.</p>
      <p>Software design and development in the real world is
continuously changing with the adoption of new software
development methods and paradigms, such as agile, to reduce the
development time from different SDLC phases and shortened
time to market. However, sustainability is currently not at the
core of the general development methodology in companies.
Sustainability as a main principle and value provides a
competitive advantage for companies and software designers
/developers but the major challenge is the lack of
understanding on how to institutionalize sustainability in software design
and development projects.</p>
      <p>Challenges
1. Very difficult to understand how to apply some of the
sustainability concepts because its new to me and my team
2. We have a challenge to find concrete examples online to
see how sustainability was applied to software project
definition especially in industry
3. It was challenging to give my staff additional task of
reading the Framework manual to understand how to apply it
It was difficult at first to understand how to explain the
different dimensions of sustainability to key stakeholders
(users) during discussion gathering requirements on how to
improve the existing system
1. The only issue is lack of examples to show how
sustainability has been used in different software requirements
elicitation at the beginning when using FSSSD but after couple of
meetings discussing about sustainability with the research
guy things became clearer.
2. Some of the research especially about sustainability in
system requirements I saw on google from some researchers
are too complex to apply
Brainstorming on how to connect the first, second and third
order impact in each of the sustainability dimensions was not
easy because each of us have different views on what is the
right thing to put but eventually we looked at some of the
examples provided by the researcher guy in using FSSSD.</p>
      <p>This paper summarizes our early results on applying the
Framework for the Sustainability of Software System Design
(FSSSD) (Figure 1 and Table 1) in two case studies. The
FSSSD provides support for sustainability in software design
through the aspect of promoting sustainability goals at each
stage of a software development life cycle phase with aid from
different sustainability concepts, tools and methods as seen in
case study 1 (Table 2) and case study 2 (Table 3 and Figure 4,
5). It also encourages a sustainability-oriented software
development mindset over time with usage of FSSSD, because
sustainability becomes part of the core fundamental values for
software design and development practice.</p>
      <p>Discussions with stakeholders and feedback in each of the
case studies (Table 4) shows the major challenge in application
of sustainability to software design and development is the lack
of readily available software system industry examples and best
practices of how core principles of sustainability are applied
and exemplified in software projects.</p>
      <p>Another challenge is in shifting developers’ mindsets to
adopting sustainability in a way that translates into their
software design and development decisions and practices. The
concept of sustainability dimensions (social, individual,
environmental, economic, and technical) only becomes interesting
to apply in software design if it can provide companies with
opportunities for cutting costs and offer a competitive
advantage in one way or another through usage of the framework.</p>
      <p>The next phase is to repeatedly apply the FSSSD to
different kinds of software projects and record best practices from
each of these projects that can then be disseminated to
interested stakeholders. Our template for documenting software
sustainability requirement elicitation best practice during software
design and development [39] can serve as template for such
documentation.</p>
    </sec>
    <sec id="sec-4">
      <title>ACKNOWLEDGMENT</title>
      <p>
        The authors would like to thank Tom Mistretta and
Alexandre Devinez for their work in the Living Lab at LUT
University to support this research work.
[1]
[
        <xref ref-type="bibr" rid="ref14">2</xref>
        ]
[3]
[4]
[5]
[6]
[23]
      </p>
      <p>G. G. Calienes, “Requirements Prioritization Framework for</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          <string-name>
            <given-names>S.</given-names>
            <surname>Oyedeji</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Seffah</surname>
          </string-name>
          , and
          <string-name>
            <given-names>B.</given-names>
            <surname>Penzenstadler</surname>
          </string-name>
          , “
          <article-title>A catalogue supporting software sustainability design,” Sustainability</article-title>
          , vol.
          <volume>10</volume>
          , no.
          <issue>7</issue>
          , pp.
          <fpage>1</fpage>
          -
          <lpage>30</lpage>
          ,
          <year>2018</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          <string-name>
            <given-names>N.</given-names>
            <surname>Shedroff</surname>
          </string-name>
          ,
          <article-title>Design is the Problem:The Future of Design Must be Sustainable</article-title>
          .
          <source>Rosenfeld Media</source>
          ,
          <year>2009</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          <string-name>
            <surname>Penzenstadler</surname>
            , and
            <given-names>C. C.</given-names>
          </string-name>
          <string-name>
            <surname>Venters</surname>
          </string-name>
          , “Sustainability Design in Requirements Engineering : State of Practice,” pp.
          <fpage>533</fpage>
          -
          <lpage>542</lpage>
          ,
          <year>2016</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          <string-name>
            <surname>Work. Requir. Eng. Sustain. Syst.</surname>
          </string-name>
          ,
          <year>2012</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          <string-name>
            <surname>U. K. Jannat</surname>
          </string-name>
          , “Green Software Engineering Adaption In Requirement Elicitation Process,” vol.
          <volume>5</volume>
          , no.
          <issue>08</issue>
          , pp.
          <fpage>94</fpage>
          -
          <lpage>98</lpage>
          ,
          <year>2016</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          <string-name>
            <given-names>United</given-names>
            <surname>Nations</surname>
          </string-name>
          , “Sustainable Development Goals Available [22] at: https://www.un.org/sustainabledevelopment/sustainabledevelopment-goals/ Accessed on 28-12-2018,” no.
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          <source>September</source>
          <year>2000</year>
          , pp.
          <fpage>8</fpage>
          -
          <lpage>23</lpage>
          ,
          <year>2015</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          <string-name>
            <given-names>S.</given-names>
            <surname>Oyedeji</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Seffah</surname>
          </string-name>
          , and
          <string-name>
            <given-names>B.</given-names>
            <surname>Penzenstadler</surname>
          </string-name>
          , “
          <source>Classifying the Measures of Software Sustainability,” in Proceedings of the 4th International Workshop on Measurement and Metrics for Green and Sustainable Software Systems co-located with 12th International Symposium on Empirical Software Engineering and Measurement (ESEM</source>
          <year>2018</year>
          ),
          <year>2018</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          <string-name>
            <given-names>G.</given-names>
            <surname>Deloittte</surname>
          </string-name>
          , “
          <source>2016 Mobile Industry Impact Report: Sustainable Development Goals</source>
          ,”
          <year>2016</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          <string-name>
            <given-names>C.</given-names>
            <surname>Calero</surname>
          </string-name>
          and
          <string-name>
            <given-names>M.</given-names>
            <surname>Piattini</surname>
          </string-name>
          , “
          <article-title>Introduction to Green in software engineering,” Green Softw</article-title>
          . Eng., pp.
          <fpage>1</fpage>
          -
          <lpage>327</lpage>
          ,
          <year>2015</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          <string-name>
            <given-names>J.</given-names>
            <surname>García-Berna</surname>
          </string-name>
          , J. Carrillo de Gea,
          <string-name>
            <given-names>B.</given-names>
            <surname>Moros</surname>
          </string-name>
          ,
          <string-name>
            <surname>J. FernándezAlemán</surname>
          </string-name>
          , J. Nicolás,
          <article-title>and</article-title>
          <string-name>
            <given-names>A.</given-names>
            <surname>Toval</surname>
          </string-name>
          , “
          <source>Surveying the Environmental and Technical Dimensions of Sustainability in Software Development Companies,” Appl. Sci.</source>
          , vol.
          <volume>8</volume>
          , no.
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          11, p.
          <fpage>2312</fpage>
          ,
          <year>2018</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          <string-name>
            <given-names>H.</given-names>
            <surname>Knut</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R.</given-names>
            <surname>Martin</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Ingridvon</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Michael</surname>
          </string-name>
          ,
          <string-name>
            <given-names>K.</given-names>
            <surname>David</surname>
          </string-name>
          , and
          <string-name>
            <given-names>K.</given-names>
            <surname>Nina</surname>
          </string-name>
          , “
          <article-title>Sustainability Nears a Tipping Point Sustainability Nears a Tipping Point,” MIT Sloan Manag</article-title>
          .
          <source>Rev.</source>
          , vol.
          <volume>53</volume>
          , no.
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          2, pp.
          <fpage>69</fpage>
          -
          <lpage>74</lpage>
          ,
          <year>2012</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref15">
        <mixed-citation>
          <string-name>
            <surname>Ericsson</surname>
            , “Energy and
            <given-names>Carbon</given-names>
          </string-name>
          <string-name>
            <surname>Report</surname>
          </string-name>
          ,” no. June, p.
          <fpage>12</fpage>
          ,
          <year>2013</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref16">
        <mixed-citation>
          <string-name>
            <surname>Microsoft</surname>
          </string-name>
          , “
          <article-title>Microsoft 2015 Citizenship Report</article-title>
          ,”
          <year>2015</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref17">
        <mixed-citation>
          <string-name>
            <given-names>S.</given-names>
            <surname>Oyedeji</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Seffah</surname>
          </string-name>
          , and
          <string-name>
            <given-names>B.</given-names>
            <surname>Penzenstadler</surname>
          </string-name>
          , “Sustainability Quantification in Requirements Informing Design,” 6th Int.
        </mixed-citation>
      </ref>
      <ref id="ref18">
        <mixed-citation>
          <string-name>
            <surname>Work. Requir. Eng. Sustain. Syst.</surname>
          </string-name>
          ,
          <source>vol. i</source>
          ,
          <year>2017</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref19">
        <mixed-citation>
          <string-name>
            <surname>Eng. Found. Softw. Qual.</surname>
          </string-name>
          , vol.
          <volume>4542</volume>
          , no.
          <source>January</source>
          , pp.
          <fpage>247</fpage>
          -
          <lpage>261</lpage>
          ,
          <year>2007</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref20">
        <mixed-citation>
          <string-name>
            <given-names>C.</given-names>
            <surname>Becker</surname>
          </string-name>
          et al.,
          <source>“Sustainability Design and Software: The Karlskrona Manifesto,” Proc. - Int. Conf. Softw. Eng.</source>
          , vol.
          <volume>2</volume>
          , pp.
          <fpage>467</fpage>
          -
          <lpage>476</lpage>
          ,
          <year>2015</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref21">
        <mixed-citation>
          <string-name>
            <given-names>B.</given-names>
            <surname>Penzenstadler</surname>
          </string-name>
          , “
          <article-title>What does Sustainability mean in and</article-title>
          for Software Engineering ?,
          <source>” 1st Int. Conf. ICT Sustain</source>
          .,
          <year>2013</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref22">
        <mixed-citation>
          <string-name>
            <given-names>B.</given-names>
            <surname>Penzenstadler</surname>
          </string-name>
          and
          <string-name>
            <given-names>H.</given-names>
            <surname>Femmer</surname>
          </string-name>
          , “
          <article-title>A generic model for sustainability with process- and product-specific instances</article-title>
          ,
          <source>” GIBSE 2013 - Proc</source>
          .
          <year>2013</year>
          Work. Green Softw. Eng.
          <article-title>Green by Softw</article-title>
          . Eng., no.
          <source>June</source>
          <year>2015</year>
          , pp.
          <fpage>3</fpage>
          -
          <lpage>7</lpage>
          ,
          <year>2013</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref23">
        <mixed-citation>
          <string-name>
            <surname>C. C. Venters</surname>
          </string-name>
          et al.,
          <source>“Software sustainability: The modern tower of babel,” 3rd Int. Work. Requir. Eng. Sustain. Syst.</source>
        </mixed-citation>
      </ref>
      <ref id="ref24">
        <mixed-citation>
          <string-name>
            <surname>Work. Proc.</surname>
          </string-name>
          , vol.
          <volume>1216</volume>
          , pp.
          <fpage>7</fpage>
          -
          <lpage>12</lpage>
          ,
          <year>2014</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref25">
        <mixed-citation>
          <string-name>
            <surname>M. Al Hinai</surname>
            and
            <given-names>R.</given-names>
          </string-name>
          <string-name>
            <surname>Chitchyan</surname>
          </string-name>
          , “Engineering Requirements for Social Sustainability,
          <source>” Proc. ICT Sustain</source>
          .
          <year>2016</year>
          ,
          <year>2016</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref26">
        <mixed-citation>
          <string-name>
            <given-names>B.</given-names>
            <surname>Penzenstadler</surname>
          </string-name>
          , “Software Engineering for Sustainability.” E. Blevis,
          <string-name>
            <given-names>C.</given-names>
            <surname>Preist</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Schien</surname>
          </string-name>
          , and
          <string-name>
            <given-names>P.</given-names>
            <surname>Ho</surname>
          </string-name>
          , “
          <article-title>Further Connecting Sustainable Interaction Design with Sustainable Digital Infrastructure Design,”</article-title>
          <source>Proc. 2017 Work. Comput.</source>
        </mixed-citation>
      </ref>
      <ref id="ref27">
        <mixed-citation>
          <string-name>
            <surname>Within</surname>
          </string-name>
          Limits - LIMITS '17, pp.
          <fpage>71</fpage>
          -
          <lpage>83</lpage>
          ,
          <year>2017</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref28">
        <mixed-citation>
          <source>[24] [25] [26] [27] [28] [29] [30] developing Green and Sustainable Software using ANP - based Decision Making</source>
          ,” pp.
          <fpage>1</fpage>
          -
          <lpage>9</lpage>
          ,
          <year>2013</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref29">
        <mixed-citation>
          <string-name>
            <given-names>B.</given-names>
            <surname>Penzenstadler</surname>
          </string-name>
          , “
          <article-title>RE4ES: Support Environmental Sustainability by Requirements Engineering</article-title>
          ,” First Int.
        </mixed-citation>
      </ref>
      <ref id="ref30">
        <mixed-citation>
          <string-name>
            <surname>Work. Requir. Eng. Sustain. Syst.</surname>
          </string-name>
          ,
          <year>2012</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref31">
        <mixed-citation>
          <string-name>
            <given-names>P.</given-names>
            <surname>Bozzelli</surname>
          </string-name>
          ,
          <string-name>
            <given-names>Q.</given-names>
            <surname>Gu</surname>
          </string-name>
          , and
          <string-name>
            <given-names>P.</given-names>
            <surname>Lago</surname>
          </string-name>
          , “
          <article-title>A systematic literature review on green software metrics</article-title>
          ,
          <source>” Sis.Uta.Fi</source>
          ,
          <year>2013</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref32">
        <mixed-citation>
          <string-name>
            <surname>Richardson</surname>
          </string-name>
          , “
          <article-title>Developing a sustainability non-functional requirements framework</article-title>
          ,
          <source>” Proc. 3rd Int. Work. Green Sustain. Softw. - GREENS</source>
          <year>2014</year>
          , pp.
          <fpage>1</fpage>
          -
          <lpage>8</lpage>
          ,
          <year>2014</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref33">
        <mixed-citation>
          <string-name>
            <surname>C. C. Venters</surname>
          </string-name>
          et al., “
          <article-title>The Blind Men and the Elephant Towards an Empirical Evaluation Framework for Software Sustainability</article-title>
          ,” vol.
          <volume>2</volume>
          , no.
          <issue>1</issue>
          , pp.
          <fpage>1</fpage>
          -
          <lpage>6</lpage>
          ,
          <year>2014</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref34">
        <mixed-citation>
          <string-name>
            <surname>Tomlinson</surname>
          </string-name>
          , “
          <article-title>Safety, security, now sustainability: The nonfunctional requirement for the 21st century,” IEEE Softw.</article-title>
          , vol.
          <volume>31</volume>
          , no.
          <issue>3</issue>
          , pp.
          <fpage>40</fpage>
          -
          <lpage>47</lpage>
          ,
          <year>2014</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref35">
        <mixed-citation>
          <string-name>
            <given-names>K.</given-names>
            <surname>Roher</surname>
          </string-name>
          and
          <string-name>
            <given-names>D.</given-names>
            <surname>Richardson</surname>
          </string-name>
          , “Sustainability requirement patterns,
          <source>” 2013 3rd Int. Work. Requir. Patterns, RePa 2013 - Proc.</source>
          , pp.
          <fpage>8</fpage>
          -
          <lpage>11</lpage>
          ,
          <year>2013</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref36">
        <mixed-citation>
          <string-name>
            <given-names>L.</given-names>
            <surname>Bonanni</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Busse</surname>
          </string-name>
          , J. Thomas,
          <string-name>
            <given-names>E.</given-names>
            <surname>Blevis</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Turpeinen</surname>
          </string-name>
          , and
          <string-name>
            <given-names>N. J.</given-names>
            <surname>Nunes</surname>
          </string-name>
          , “Visible - Actionable - Sustainable: Sustainable Interaction Design in Professional Domains,
          <source>” Proc. CHI</source>
          <year>2011</year>
          , pp.
          <fpage>2413</fpage>
          -
          <lpage>2416</lpage>
          ,
          <year>2011</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref37">
        <mixed-citation>
          <string-name>
            <given-names>J.</given-names>
            <surname>Froehlich</surname>
          </string-name>
          ,
          <string-name>
            <given-names>L.</given-names>
            <surname>Findlater</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Landay</surname>
          </string-name>
          , and C. Science, “
          <article-title>The Design of Eco-Feedback Technology</article-title>
          ,” pp.
          <fpage>1999</fpage>
          -
          <lpage>2008</lpage>
          ,
          <year>2010</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref38">
        <mixed-citation>
          <string-name>
            <given-names>A.</given-names>
            <surname>Spagnolli</surname>
          </string-name>
          et al., “
          <article-title>Eco-feedback on the go: Motivating energy awareness,” Computer (Long</article-title>
          . Beach. Calif)., vol.
          <volume>44</volume>
          , no.
          <issue>5</issue>
          , pp.
          <fpage>38</fpage>
          -
          <lpage>45</lpage>
          ,
          <year>2011</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref39">
        <mixed-citation>
          <string-name>
            <given-names>R. J.</given-names>
            <surname>Yun</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Aziz</surname>
          </string-name>
          , and
          <string-name>
            <given-names>B.</given-names>
            <surname>Lasternas</surname>
          </string-name>
          , “
          <article-title>Design Implications for the Presentation of Eco- feedback Data,”</article-title>
          <string-name>
            <surname>Arch. Des. Res.</surname>
          </string-name>
          , vol.
          <volume>28</volume>
          , no.
          <issue>4</issue>
          , pp.
          <fpage>95</fpage>
          -
          <lpage>106</lpage>
          ,
          <year>2015</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref40">
        <mixed-citation>
          <string-name>
            <surname>Winckler</surname>
          </string-name>
          , Interaction - INTERACT
          <source>2013 14th IFIP TC 13 International Conference, no. 1</source>
          .
          <year>2013</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref41">
        <mixed-citation>
          Available at: http://dx.doi.
          <source>org/10.1016/b978-0-12-387795- 6</source>
          .
          <fpage>00001</fpage>
          -
          <lpage>9</lpage>
          .
          <year>2012</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref42">
        <mixed-citation>
          <string-name>
            <given-names>E.</given-names>
            <surname>Blevis</surname>
          </string-name>
          , “Sustainable Interaction Design : Invention &amp; Disposal , Renewal &amp; Reuse,” pp.
          <fpage>503</fpage>
          -
          <lpage>512</lpage>
          ,
          <year>2007</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref43">
        <mixed-citation>
          <string-name>
            <given-names>D.</given-names>
            <surname>Fallman</surname>
          </string-name>
          , “Design-oriented Human - Computer Interaction,” no.
          <issue>5</issue>
          , pp.
          <fpage>225</fpage>
          -
          <lpage>232</lpage>
          ,
          <year>2003</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref44">
        <mixed-citation>
          <string-name>
            <surname>“Strongly Sustainable Business Model</surname>
          </string-name>
          Canvas Http://www.ssbmg.com/ Accessed on 18-04-
          <year>2019</year>
          .”
          <string-name>
            <given-names>S.</given-names>
            <surname>Oyedeji</surname>
          </string-name>
          and
          <string-name>
            <given-names>B.</given-names>
            <surname>Penzenstadler</surname>
          </string-name>
          , “Karlskrona Manifesto:
          <article-title>Software requirement engineering good practices</article-title>
          ,
          <source>” CEUR Workshop Proc.</source>
          , vol.
          <volume>2223</volume>
          , pp.
          <fpage>15</fpage>
          -
          <lpage>23</lpage>
          ,
          <year>2018</year>
          .
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>