=Paper=
{{Paper
|id=Vol-2382/ICT4S2019_paper_1
|storemode=property
|title=Using Participatory Technical-action-research to Validate a Software Sustainability Model
|pdfUrl=https://ceur-ws.org/Vol-2382/ICT4S2019_paper_1.pdf
|volume=Vol-2382
|authors=Nelly Condori Fernandez,Patricia Lago,Miguel Luaces,Angeles Saavedra,Leticia Gonzalez
|dblpUrl=https://dblp.org/rec/conf/ict4s/Condori-Fernandez19
}}
==Using Participatory Technical-action-research to Validate a Software Sustainability Model==
https://ceur-ws.org/Vol-2382/ICT4S2019_paper_1.pdf
Using Participatory Technical-action-research
to validate a Software Sustainability Model
Nelly Condori Fernandez∗† Patricia Lago∗ , Miguel R. Luaces† , Ángeles S. Places† , Leticia González Folgueira‡
∗ Vrije Universiteit Amsterdam, The Netherlands
n.condori-fernandez@vu.nl, p.lago@vu.nl
† Universidade da Coruña, Database Laboratory, Spain
n.condori.fernandez@udc.es, miguel.luaces@udc.es, angeles.saavedra.places@udc.es
‡ Enxenio, Spain
lgonzalez@enxenio.es
Abstract—[Context and Motivation] In the last years, Lago et al. [8] and Venters et al. [7] agree on
software engineering researchers have contributed to defining software sustainability in terms of multiple and
defining the notion of sustainability-aware software as interdependent dimensions (e.g. economic, technical,
a quality requirement. [Question/problem] The field is, social, environmental, individual). However, despite this
however, still missing instruments supporting the design multidimensional nature of sustainability, most of the
and assessment of software sustainability.
current efforts have been put on understanding what
[Objective] This research aims at providing a validated
Sustainability Assessment Framework (SAF) through a software aspects can impact on the environmental sus-
long-term empirical study in close-collaboration with the tainability dimension [13].
software industry. [Methodology] By using the partici-
patory technical action-research method, we validate the In order to provide a better characterization of the
sustainability-quality model, one of the instruments of the sustainability dimensions, Condori-Fernandez and Lago
SAF framework, by means of investigating its applicability [9] proposed a preliminary sustainability-quality model
in an industrial software project and detecting potential for software-intensive systems. It consists of quality at-
improvements. tributes that contribute to each sustainability dimension,
[Results] Our results confirm the effective applicability of
our model as most of its quality attributes (QAs) have been
and their corresponding direct dependencies. According
either addressed in the software project or acknowledged to the authors [9], software sustainability is defined in
as relevant. The action-research method was also very terms of four dimensions: the social dimension aims to
useful for enriching our model by identifying QAs missing allow current and future generations to have equal and
in the model (e.g. regulation compliance, data privacy). equitable access to the resources in a way that preserves
[Contribution] The sustainability-quality model can their socio-cultural characteristics and achieve healthy
be effectively used as an instrument for identifying and modern society. The environmental dimension seeks
sustainability-quality requirements, and creating aware- to avoid that software-intensive systems harm the en-
ness on the relevance of the multidimensional sustainabil- vironment they operate in. The technical dimension is
ity nature of certain quality attributes. concerned with supporting long-term use and appropri-
Keywords—sustainability-quality model, quality require- ate evolution/adaptation of software-intensive systems
ments, action research, software-intensive systems. in constantly changing execution environment. And,
the economic dimension aims to ensure that software-
intensive systems can create economic value. It is taken
I. I NTRODUCTION
care of in terms of budget constraints and costs as
In recent years, the concept of sustainability has well as market requirements and long-term business
been recognized as i) a timely important concern for objectives that get translated into requirements for the
researchers from different disciplines in computing (e.g. system under consideration.
artificial intelligence [1], human computer interaction
[2], software engineering [3], [4]; and ii) a key driver As shown in Figure 1, the Sustainability-quality
of innovation at software companies [5]. Thanks to this model is one of the key instruments of the Sustain-
growing acknowledgment, several efforts have been put ability Assessment Framework (SAF) proposed by the
for understanding what software sustainability means authors and consisting of three main components: the
(e.g. [6], [7], [8], [9] ) and how software engineering sustainability-quality model [9], [14], the architectural
can support sustainability (e.g. [10], [11], [12]]). decision map [15], and the metrics [16].
In order to validate SAF in close collaboration with and incremental research cycles are carried out at the
software industry, we have designed a long-term em- product level, where each increment corresponds to
pirical study supported by action-research methodology the instruments of the SAF framework (Sustainability-
[17], [18]. In this paper, we present the overall overview quality model, the Decision maps, and the Metrics
of our empirical research strategy, as well as the first catalogue). Finally, the results emerging from the stage
validation results as a consequence of applying the Reflection (at the product level) are fed back to the
sustainability-quality model in a software project that participating software company (at the product-family
focuses on the achievement of sustainability goals. level).
Notice that the involvement of any other project
from the product-family in the subsequent iterations
(action, feedback collection and reflection) is allowed
as long as the participants agree as part of the diagnosis
stage.
The following subsections present the research ques-
tions shaping our study, the empirical research context
(including the project and participants) and the unit of
Figure 1: Timeline of the research cycles applied to analysis.
validate the SAF framework
A. Research Questions
In order to validate the SAF framework, the follow-
The following sections provide a detailed account ing research questions are formulated:
of our study. Section 2 describes the research method.
Section 3 reports our main findings. Then section 4 (RQ1) How applicable is the SAF framework to
discusses the validity threats and section 5 the related assess the software architecture at hand?
works. Section 6 concludes the paper and discuss further (RQ2) Which improvements to the SAF frame-
work. work are realized as an outcome of imple-
menting the action research?
II. R ESEARCH METHOD
In this paper, we focus on answering both research
Our research strategy is inspired by the combi- questions with a special focus on the sustainability-
nation of two types of action research: (i) technical quality model as illustrated in Figure 1. As such, the an-
action research because we aim to scale up the action swer to RQ1 aims at (i) identifying sustainability-quality
(treatment) to conditions of practice by actually using attributes present in the model and already addressed in
it in a particular problem [17]; and (ii) participatory the selected project, and (ii) discovering sustainability-
because the researchers are active in making informed quality requirements that are not yet included in the
decisions throughout all aspects of the research process, model but are relevant for the selected project.
as the participating organization shares experiences of
applying the action [18], [19]. According to Petersen et The initial sustainability-quality model proposed by
al. [19], the action is the treatment introduced by the Condori and Lago [9], [14] was used as the starting
researcher to induce a positive change in the company. point for answering RQ1.
In our study, the action is the SAF framework. Naturally, its iterative application to the selected
The research strategy we use to validate the SAF products can enrich the model itself with new insights
framework consists of four stages (grey-colored in and/or new attributes, and hence help answering RQ2.
Figure 2): diagnosis, action, feedback collection and Figure 3 summarizes the possible states that can be
reflection. These stages are carried out at two different determined as a result of applying the sustainability-
levels. We start the diagnosis at the product-family quality model: Quality Attribute (QA) discovered (or-
level with the purpose of understanding the common ange cell), QA covered (green cell), QA missing (red
characteristics of the family of products and identifying cell), and n/a when a QA is not observable (grey cell).
sustainability-related issues. The output of the diagnosis These states will be used in Section III and Tables II
is the selection of a product (software project), which is and III to present the study results.
then used in the following three stages. At the product
level, we plan the research to apply the SAF to the
B. Research context
selected software project, then we collect the feedback
from the participants and reflect on it to refine the design The research context is characterized by six software
of the SAF framework. As shown in Figure 1, iterative products developed by the Database Laboratory (LBD),
2
Figure 2: Participatory Technical-Action Research Process
a research group of the University Of A Coruña, as them acted as responsible for planning and executing
part of the GIRO Project1 . A spin-off software company, the application of the SAF framework. Both participants
Enxenio (ENX), will maintain the software products in are researchers, who played three different roles in the
the future. action research [17]: (i) Designer: designing the instru-
ments of the SAF framework; (ii) Helper: using the
1) The GIRO project: The GIRO Project, funded by instruments of the SAF framework to help the software
the FEDER Interconnecta program, has a reference ar- company in getting awareness on the sustainability-
chitecture, reused and adapted to addresses the specific quality requirements that were addressed in the project;
requirements of the individual customer companies. and (iii) Researcher: drawing lessons learned about the
The customer companies of the GIRO consortium instruments of the SAF framework.
are leaders in the following market areas: Two participants were practitioners from the
• Company A: treatment of meat by-products not Database Laboratory and the software company (Enx-
intended for human consumption, particularly enio), who used the sustainability-quality model to
collecting dead animals in the northwest of identify relevant QAs that had not been considered in
Spain. the project, as well as missing QAs that were not present
in the Model. The roles of these participants were
• Company B: management and valorization of software analyst and software project manager. Besides
organic waste by means of its transformation the GIRO project counted with a software architect,
in biogas. the third practitioner from Enxenio, who contributed
solving some doubts in the QAs identification process.
• Company C: installation and management of
alarm systems in companies and domiciles.
C. Unit of analysis and Procedure
• Company D: health and well-being services
The unit of analysis of the study is the prod-
offered for elderly people residing in their own
uct family developed under the GIRO project, whose
home.
documentation was analyzed and discussed in several
• Companies E and F: prevention of occupational meetings as part of the diagnosis stage. The software
risks regarding health at work. company is interested in knowing how sustainable is
the reference software architecture used for the GIRO
2) Participants: The participatory-technical action product family.
research team consisted of five participants. Two of
In this paper, we focus on the software product re-
1 Acronym used for the project name: ”Generating, Managing and quired by the company A, which is a Mobile Workforce
Integrating Routes using OLAP” Management (MWM) System developed to support the
3
Table I: Focus groups conducted during the first research III. R ESULTS AND D ISCUSSION
cycle A. RQ1: Applicability of the model
FG Purpose LBD/ENX ENX The following presents our study results. They refer
FG1 Social sustainability 2
to Table II for the QAs already covered by the Model
FG2 Technical sustainability 2 1 (i.e. addressed in the project or relevant but not yet
FG3 Economic-Environmental sustainability 2 addressed), and Table III for the new QAs that were
FG4 Generic 1 1
missing in the Model.
1) Quality attributes covered in the project: These
dead animal collection. Its selection was as a result are:
of the activities carried out at the diagnosis stage. In Compatibility. This QA in terms of interoperability
order to answer our research questions, the next re- and co-existence was addressed in the project since
search activities (action planning and -taking, feedback there was a need of sharing information with some
collection and reflection) were carried out at the product existing systems (i.e. Libra, active directory) used by
level. Through an action plan defined for allowing the customer company. The relevance to the social
the application of our sustainability-quality model [14] sustainability dimension was acknowledged because of
(action/treatment) to the selected product, we carried the greater access of information (from different sys-
out action taking (i.e. technical-documentation analysis tems) that can then be used by users (e.g. administrator,
and four focus group meetings [20]). Each focus- planner). Moreover, the compatibility’s relevance to
group meeting was planned by the researchers (the first the technical dimension was also confirmed since as
author played the moderator’s role). The purpose of the long as the MWM system performs efficiently, while
corresponding focus groups and participants are shown both environment and resources are shared with other
in Table I. Although these focus groups were small, the software system, its use will last longer.
participants reflected on the analyzed QAs, by explain-
ing their relevance for the selected project and giving Effectiveness. This QA was addressed in the project
examples on how some of the QAs were addressed. and its relevance to the technical, social and economic
Because of this active discussion among participants dimensions was confirmed. Thanks to the 1-month
(practitioners and researchers), we considered the four testing period, developers found that the MWM system
sessions as a focus group and not an interview. Each provides a good support to accurately perform user
focus group had an average duration time of 60 minutes, tasks that achieve all specified stakeholders goals such
and notes were taken during the focus group meetings as: the planner is able to schedule new requests from
by the moderator (feedback collection). The following the insurance company; the driver is able to store/track
section presents the results obtained from the reflection requests until such time as they are complete, etc. It is
on the collected feedback to contribute to the answers technical because effectiveness contribute to the long-
to RQ1-RQ2. term usage of the MWM system. It is social because
of the accurate performance of all user tasks contribute
to their own well-being (e.g. drivers less stressed using
the MWM system for tracking new requests). Moreover,
the achievement of stakeholders goals contribute to one
of the long-term business objectives of the software
company (i.e. customer satisfaction), as well as the
saving costs for fixing failures (economic dimension).
Efficiency. Relevance of Efficiency to the environ-
ment dimension is confirmed and also partially ad-
dressed, by having a good usage of certain resources
when users perform their tasks supported by the system.
For instance, the average time used for completing
user tasks (e.g. administrator/planner for scheduling
new requests, drivers register attended requests) was
according to the expectations of the customer company.
However, other type of resources, like the amount of
energy, used by the MWM system when users perform
Figure 3: Specific research questions investigated certain tasks, has not been yet evaluated. Similarly to
through the action research effectiveness, this QA also contributes to the economic
dimension because of the saving costs for fixing failures.
4
Freedom from risk. It is addressed in terms of: Performance efficiency. This requirement in terms
environmental risk mitigation, by means of the timely of time behaviour was acknowledged as a good contrib-
use of the MWM system, which aims to provide support utor to both technical and environmental dimensions.
for collecting dead animals under the European regula- While the system shows an efficient performance (e.g.
tions (CE 999/2001). By doing so, the MWM system processing and response time) in delivering the main
helps (i) avoid exposing people to potential disease- functionalities (e.g. finding optimum route, allocating
causing pathogens (social dimension); and (ii) reduce new requests in the planning), its use will last longer
environmental concerns like potential contamination of (technical sustainability dimension).
air, soil, surface and sub-surface water (environmental
Reliability. There was a consensus that the MWM
dimension). In a similar way, the relevance of Health
system must be reliable when perform not only un-
and safety risk mitigation to the social dimension was
der normal operations (maturity), but also whenever it
confirmed. The health risks to people like farmers and
is required by end-users (availability). Although both
potential meat consumers, can be mitigated by means
QAs contribute to the technical dimension, a greater
of the timely use of the MWM system. Regarding
availability of the MWM system (e.g. visualization
the safety risk mitigation, this QA has not been yet
of the route planning, tracking driver behaviour) will
addressed for certain type of stakeholders that directly
contribute not only to the environmental and economic
interact with the MWM system. For instance, for drivers
sustainability dimensions, but also to the social dimen-
who are exposed to road safety risks, the system could
sion. It is social because the environmental and social
allow an adaptive multimodal interaction to show the
sustainability goals of this kind of systems can be
route for reaching a target place (e.g. a farm).
achieved only if the software services are available. The
Functional Suitability. This quality is addressed social contribution was marked with an “+” since it was
in terms of functional appropriateness and functional not considered in the original model.
correctness. Practitioners confirmed the contribution of Satisfaction. This requirement in terms of trust
both QAs to the technical and economic sustainability and usefulness is relevant to address both social and
dimensions. economic sustainability dimensions. As practitioners
Both quality requirements were verified through one- consider that the direct relation between user satisfac-
month testing. All the functionalities implemented as tion and the technology acceptance( [21], [22]) has a
part of the MWM system facilitate the accomplishment positive impact on the social sustainability dimension
of tasks performed by the planner, driver, and system ad- since satisfied users will be in a much better position for
ministrator. For example, the planner, main component getting access to social resources provided by the corre-
of the MWM system, consider all relevant parameters sponding software system. Also, the practical relevance
to provide an optimum route planning to be used by the of both QAs to the economic dimension was confirmed
drivers. since customers satisfaction is considered as one of the
Functional suitability is relevant to the technical dimen- primary business objectives of the software company.
sion since both QAs contribute to the long-term use of Moreover, as long as the usefulness and trust of the
the system. It is also relevant to the economic dimension system are valued by the end-users, the acceptance to
because the software company do not have to dedicate use the system will be prolonged, which means that both
much effort on corrective maintenance actions. QAs contribute to the technical dimension as well.
Maintainability. The relevance of this requirement Security. This requirement in terms of accountabil-
is acknowledged with respect to the modifiability, modu- ity, authenticity, confidentiality and integrity was con-
larity and reusability attributes. As a common architec- sidered as a good contributor to the social sustainability.
ture is planned to be used for the six software systems Confidentiality was addressed thanks to the Role Based
of the GIRO project, modifiability and modularity are Access Control implementation, where roles previously
key to facilitate the adaptation of the reference software defined are assigned only to group of users that are
architecture to be used in the product family and its authorized to have access to the system. Authenticity
evolution (technical dimension). was also addressed since the login of each user has
Moreover, as the company aims to create a flexible been used as identifier. And as several user actions
architecture for addressing all relevant requirements of can be traced, accountability was also addressed but
each software system of the product family, modifia- only for certain actions (i.e. updates of new routes).
bility and reusability should contribute also to reduce Regarding the integrity of the system, this requirement
redesign costs and allow quicker response to company was also considered as a good contributor to the tech-
customers. However, as the relevance of both QAs to the nical dimension because the MWM system is able to
economic dimension was not identified in the original prevent (in certain extent) unauthorized access to the
model, in Table II we mark both contributions with an system, and does so to the extent that security controls
“+”. specified for that system cannot be compromised. This
5
new contribution was also marked with an “+”. Testability. It is another relevant attribute that was
acknowledge as relevant for the technical sustainability
Usability. As expected usability attributes like ap-
dimension, but it was not addressed since we could not
propriateness recognizability, operability, and user error
find any evidence on how testable are the software arte-
protection were clearly considered as relevant to the so-
facts (MWM system, module) to find (critical) faults.
cial sustainability dimension. User error protection was
addressed through the use of field-validation methods, Capacity. This QA was acknowledged as relevant
mandatory fields, and action confirmation. Protecting for the technical dimension but not addressed. Load or
users against making errors it is social because it stress testing could have been carried out for determin-
contributes not only to satisfy usability requirements but ing the weak points of the system architecture.
also to the quality of user experience [23]. However,
this QA was also considered as good contributor to Accessibility. Although the company has knowl-
both technical and economic dimensions, which were edge on existing standard accessibility guidelines (e.g.
also confirmed by two case studies. As both contribu- ISO/IEC 40500:2012), this QA was not addressed in
tions were not considered in the original model, they the project since was not considered as very relevant
are marked with an “+”. During the testing period, for the project. However, participants agreed that imple-
issues related to operability of the system were fixed. menting some accessibility features that can help certain
Moreover, as end-users recognized that the system is users like ”deaf and hearing impaired drivers” could
appropriate for meeting their corresponding needs (i.e. be beneficial to our society in long term, by removing
drivers need to attend all assigned requests per day), interaction/communication barriers.
appropriateness recognizability was also addressed.
Learnability. Although this QA was not imple-
2) Quality Attributes discovered in the project: In mented, its relevance was acknowledged not only for
this section, we discuss the QAs that were discovered the social dimension but also for the economic dimen-
as relevant for the selected project as a result of using sion. By implementing features that aid novice users
the sustainability-quality model. to quickly learn and also allow steady progression to
expertise, it could results on saving training costs to
Context Coverage. Practitioners agreed on the rel-
the company. (This new contribution to the economic
evance of addressing context coverage requirement in
dimension is marked with an “+”).
terms of context completeness and flexibility. Despite
issues related to effectiveness and efficiency were veri- Robustness. Given that the MWM system could
fied during the one-month testing period, it is possible be affected by several unexpected situations (e.g. GPS
that some contexts where the system could be used were signal is lost), robustness is consider as relevant to
not covered by the testing process. The need of explicit address the technical sustainability dimension.
specifications about the different contexts of use was
acknowledged as a relevant requirement that should be
considered as good contributor to both technical and B. RQ2: Improvements in the model
economic dimensions. Moreover, if the system is not
able to work at any other potential context of use (not This section reports on the missing QAs that were
explicitly specified), a higher effort and costs would be identified as relevant to be included in the Model as well
needed to improve the flexibility of the MWS system. as their corresponding contributions to the sustainability
dimensions (see Table III):
Modifiability and Reusability. Although both
maintainability attributes were considered as relevant to 1) Quality attributes missing in the Model: These
contribute to the environmental dimension, they were are:
not addressed because of the lack of facilities/tools for
determining in which extent the software artefacts (e.g.
Data Privacy. Given MWM system needs data to be
modules) of the system that are modifiable or reusable
stored and shared for enabling the management of work
contribute to reduce environmental impact.
assignments and the tracking of real-time field workers,
Resource utilization. This requirement was discov- data privacy is a key requirement that should be con-
ered as relevant to the project and its contribution to sidered as good contributor to the social sustainability
the environmental dimension was confirmed. However, dimension. Timeliness. With MWM systems, timeliness
despite the contribution of the planner module to the op- of information is needed for allowing field workers to
timization of resources utilization(e.g. trucks needed for continue with their job successfully (rapid collection of
collecting dead animals, gasoline consumed by trucks), dead animals). As this QA concerns on the favourable
this requirement was not fully addressed because of the time of having the right information(a social resource),
lack of a manager that could help in determining the it has been also considered as a good contributor to the
actual amount of resources used by the system. social sustainability.
6
Table II: Sustainability-quality analysis of the MWM system (Green cell= QA is addressed, orange cell= QA is
discovered as relevant, light-gray cell= QA is in the model but not relevant for the project, += new contribution)
Characteristics Attributes Definition according to [9] TECH SOC ENV ECON
Co-existence product can perform its functions efficiently while sharing environment and
Compatibility
resources with other products.
Interoperability a system can exchange information with other systems and use the information
that has been exchanged.
Context cov- Context com- system can be used in all the specified contexts of use
erage pleteness
Flexibility system can be used in contexts beyond those initially specified in the require-
ments.
Effectiveness Effectiveness accuracy and completeness with which users achieve specified goals.
Efficiency Efficiency resources expended in relation to the accuracy and completeness with which
users achieve goals.
Freedom Economic risk system mitigates the potential risk to financial status in the intended contexts of
from risk mitigation use.
Environmental system mitigates the potential risk to property or the environment in the intended
risk mitigation contexts of use.
Health and system mitigates the potential risk to people in the intended contexts of use.
safety risk
mitigation
Functional Functional ap- the functions facilitate the accomplishment of specified tasks and objectives.
suitability propriateness
Functional system provides the correct results with the needed degree of precision.
correctness
Functional degree to which the set of functions covers all the specified tasks and user
completeness objectives.
Maintainability Modifiability system can be effectively and efficiently modified without introducing defects +
or degrading existing product quality
Modularity system is composed of components such that a change to one component has
minimal impact on other components.
Reusability an asset can be used in more than one system, or in building other assets +
Testability effectiveness and efficiency with which test criteria can be established for a
system.
Performance Capacity the maximum limits of a product or system parameter meet requirements.
efficiency
Resource uti- the amounts and types of resources used by a system, when performing its
lization functions, meet requirements.
Time response, processing times and throughput rates of a system, when performing
behaviour its functions, meet requirements.
Portability Adaptability system can effectively and efficiently be adapted for different or evolving
hardware, software or usage environments.
Replaceability product can be replaced by another specified software product for the same
purpose in the same environment.
Reliability Availability system is operational and accessible when required for use. +
Fault tolerance system operates as intended despite the presence of hardware or software faults.
Maturity system meets needs for reliability under normal operation.
Recoverability system can recover data affected and re-establish the desired state of the system
is case of an interruption or a failure.
Satisfaction Trust stakeholders has confidence that a product or system will behave as intended.
Usefulness user is satisfied with their perceived achievement of pragmatic goals.
Security Accountability actions of an entity can be traced uniquely to the entity.
Authenticity the identity of a subject or resource can be proved to be the one claimed.
Confidentiality system ensures that data are accessible only to those authorized to have access.
Integrity system prevents unauthorized access to, or modification of, computer programs
or data.
Usability Appropriateness users can recognize whether a system is appropriate for their needs, even before
recognizability it is implemented.
Learnability system can be used to achieve specified goals of learning to use the system. +
Operability system has attributes that make it easy to operate and control.
User error pro- system protects users against making errors. + +
tection
Accessibility Accessibility system can be used by people with the widest range of characteristics and
capabilities.
Robustness Robustness Refers to the capability of the sytem to behave in an acceptable way in
unexpected situations
Survivability Survivability The degree to which a system continues to fulfil its mission by providing
essential services in a timely manner in spite of the presence of attacks
Regulation compliance. As the main business re- quirement of the customer company is to get that dead
7
animal removal requests can be attended without delays • by adding new QAs that had not been consid-
according to the European regulations, the software ered in the model. As shown in Table III, many
designers/developers should be aware on them to meet of these QAs were added to the social sustain-
this requirement. Regulation compliance can be relevant ability dimension (i.e. data privacy, timeliness,
for contributing to i) social sustainability since health regulation compliance, tailorability). Most im-
risks are minimized, and ii) environmental sustainability portantly only one QA (tailorability) was added
since potential contamination of natural resources (e.g. for the technical dimension. Overall, this result
water, air) is reduced. shows how important can be to make explicit
which sustainability dimension is relevant for
Scalability. As the customer company could be which QA, so that significant metrics can be
interested in scaling up number of clients(currently col- identified and monitored.
lection services are offered only at the Galician region),
this would have an impact on the architecture because it • by uncovering new direct dependency relations2
should meet efficiently the increased workload as well. as a consequence of identifying new contribu-
Thus, scalability is considered as a relevant QA for tions. The QAs that were included in this type
the economic sustainability dimension because of the of relations are shown in Table II, whose cells
significant costs saving. are marked with an “+”. For example, as modi-
fiability and reusability are new contributors to
Tailorability. Enabling new configuration of func-
the economic dimension, the direct dependency
tionality as well as control information provision con-
between environmental and economic dimen-
tributes to the technical and social sustainability di-
sions consists of four ordered pairs (whose
mensions. It is technical because the environments in
QAs are efficiency, availability, modifiability,
which users can both interact and engage with software
and reusability).
can contribute to the long-term usage. And it is social
because giving users the tailoring capability in their own Finally, we argue that if a QA is found to contribute to
context of use can contribute to get a better access of multiple sustainability dimensions, its definition should
the information provided by the system. be specialized for each dimension: in this way, the
specific contribution is made explicit and hence helps
C. Discussion identifying the best-fitting influencing factors, and met-
This study focused on validating the sustainability- rics. This observation addresses both RQ1 (cf. model
quality model by means of investigating its applicability applicability) and RQ2 (enriching the model).
in a real software project (RQ1) and detecting related
improvements (RQ2). IV. T HREATS TO VALIDITY
The results shown in Table II confirm the effec- Here we discuss the threats to the validity of the
tive applicability of our sustainability-quality model action research study [19] and provides rationale for
(RQ1) because of the following facts: (i) the quality related design decisions.
requirements addressed in the project are covered by
most of the QAs present in the model in the indicated Internal validity. Action research is highly context
sustainability dimension/s (cells coloured in green); (ii) dependent. To mitigate this threat, the design of our
the participants were able to become aware of the study considered a family of software products that
relevance of certain QAs that were already present in are from different application domains, which is useful
the model but that had not been addressed in the project to analyze the sensitivity of the SAF framework in
with respect to a certain sustainability dimension (cells different contexts.
coloured in orange). Construct validity. Action research is subjective as
We also found that many QAs (e.g. co-existence, the results highly depend on the reflection of the action
efficiency, availability, reusability, modifiability, trust, researcher. Several biases may occur due to: (i) two
usefulness) contribute to more than one sustainability participants are also part of the software company and
dimension of the model. This finding confirms the they could not have provided an objective/external view
multi-dimensional nature of sustainability (one of the of the situation. (ii) The researchers as the designers of
principles of the Kaslskrona Manifesto [24]) and the the action may interpret the results positively (selective
relevance level of the QAs that can be considered by bias) when reporting the results. Both issues were
software engineers when performing certain activities partially solved by involving multiple practitioners in
like design, assessment, and prioritization [9]. the iterative discussions. In this first research cycle, a
With respect to RQ2, the application to the MWM 2 A direct dependency is defined as a finite set of ordered pairs of
system helped us enrich our model in two ways: QAs, which is reflexive, symmetric and transitive [9].
8
Table III: New quality attributes and corresponding contributions to the sustainability dimensions
Characteristics Attributes Definitions TECH SOC ENV ECON
Data Privacy Data Privacy privacy concerns arise wherever personally identifiable information is collected,
stored, or used.
Timeliness Timeliness the fact or quality of being done or occurring at a favourable or useful time.
Regulation Regulation allows to draw conclusions about how well software adheres to application
compliance compliance related regulations in laws.
Scalability Scalability the ability of a computing process to be used or produced in a range of
capabilities
Tailorability Tailorability system’s capability to allow users to create or enable new configuration of
functionality as well as control information provision.
third practitioner was involved in some of the meet- which helps scoping architectural concerns and quality
ings. Moreover, the researchers reviewed carefully the requirements along the four dimensions above.
existing technical documentation to triangulate the data
Becker et al. [30] have a similar approach but
collected from the focus-group meetings.
grounded in requirements engineering instead. In ad-
External validity. The action is implemented in a dition to the above four sustainability dimensions, they
specific social setting, which can hinder the general- add the individual as a fifth sustainability dimension.
ization of the results. However, we could apply our We argue that the social dimension and the individual
findings to other projects with similar characteristics. dimension share the same social nature. Differently,
Moreover, the transferability within the setting studied the first takes a broader perspective (e.g. organizations,
may be high if the context is similar. On the other hand, society, stakeholder types). This is especially relevant in
as our initial sustainability-quality model was defined software architecture, which aims at capturing “the big
based on the ISO/IEC 25010 standard [25], we consider picture”. The second dimension, instead, is appropriate
that its generality could be more easily adaptable to whenever the concerns of the individual (e.g. end-
other domains, such as those that were considered by user, citizen) should be addressed. This naturally comes
the GIRO project. forward more frequently in requirements engineering
and human-computer interaction.
V. R ELATED W ORK
VI. C ONCLUSIONS AND F URTHER W ORK
Being able to identify the relevant quality require-
ments on sustainability is the first step towards develop- The present empirical study was designed to validate
ing software-intensive systems that fulfill sustainability the Sustainability Assessment Framework (SAF) within
concerns by design [9]. an action-research setting. This paper focused mainly on
the sustainability-quality model, one of the instruments
Venters et al. [26] discussed the notion of software of the SAF framework, that has been applied in one
sustainability based on the analysis of the literature. of the software products developed under the GIRO
After debating if it should be considered as a non- project. As a result of this application, from a prac-
functional requirement or an emergent property, the titioner perspective (RQ1), the proposed sustainability-
authors conclude it to be a multi-faceted concept and quality model was found as a useful instrument for (i)
argue for a quantitative approach. identifying the relevance level of QAs that contribute to
different sustainability dimensions (e.g. trust, modifia-
Based on the ISO/IEC 25010 Standard, Calero et
bility, efficiency), and (ii) discovering quality require-
al. [6] provide a preliminary discussion of which qual-
ments that had not yet been addressed in the project
ity characteristics should be considered in addressing
at hand (e.g.context-completeness, flexibility, testability,
software sustainability. As a next step, they propose
capacity). From a researcher perspective (RQ2), the
the definition of a quality model where sustainability
study has helped uncovering new missing QAs that were
is part of the quality of software products. In contrast
identified as relevant to be included in the sustainability-
to our work, Calero et al. defined sustainability only in
quality model (e.g. regulation compliance, data privacy).
terms of energy consumption, resource optimization and
perdurability (reusability, modifiability, and adaptabil- As a further work, we plan to apply the decision
ity). Originated in the 2013 GREENS workshop [27], maps and the metrics, by using the same software prod-
Lago et al. [8] defined a four-dimensional model that uct, and replicate the validation of the sustainability-
extends the social, environmental and economic dimen- quality model, by involving a new GIRO software
sions (rooted in the Brundtland report [28]) with the product within the same action-research environment.
technical dimension. Later on, Lago introduced the Soft- Our sustainability model will be also enriched with the
ware Sustainability Assessment (SoSA) method [29], findings obtained from the case study reported in [31].
9
ACKNOWLEDGMENT [16] N. Condori-Fernández, A. Bagnato, and E. Kern, “A focus
group for operationalizing software sustainability with the
This work has received partial support by MEASURE platform,” in 4th International Workshop on Mea-
Xunta de Galicia / FEDER-UE CSI (ED431G/01, surement and Metrics for Green and Sustainable Software
ED431C 2017/58), and MINECO-AEI/FEDER-UE Systems (MeGSuS), Co-located at ESEM, Oulu, Finland, 2018.
with the projects: Datos 4.0 (TIN2016-78011-C4- [17] R. Wieringa and A. Moralı, “Technical action research as a
1-R), ETOME-RDFD3 (TIN2015-69951-R), and validation method in information systems design science,” in
Design Science Research in Information Systems. Advances
FLATCITY(TIN2016-77158-C4-3-R.) in Theory and Practice, (Berlin, Heidelberg), pp. 220–238,
Springer Berlin Heidelberg, 2012.
R EFERENCES [18] C. MacDonald, “Understanding participatory action research:
[1] M. De-Arteaga, W. Herlands, D. B. Neill, and A. Dubrawski, A qualitative research methodology option,” Canadian Journal
“Machine learning for the developing world,” ACM Trans. of Action Research, vol. 13, no. 2, p. 34–50, 2012.
Manage. Inf. Syst., vol. 9, pp. 9:1–9:14, Aug. 2018. [19] K. Petersen, Ç. Gencel, N. Asghari, D. Baca, and S. Betz,
[2] O. Bates, V. Thomas, C. Remy, L. Nathan, S. Mann, and “Action research as a model for industry-academia collabora-
A. Friday, “The future of hci and sustainability: Championing tion in the software engineering context,” in ACM International
environmental and social justice,” in CHI EA ’18 Extended Workshop on long-term Industrial collaboration on Software
Abstracts of the 2018 CHI Conference on Human Factors in Engineering (WISE), pp. 55–62, 2014.
Computing Systems, vol. 2018-April, ACM, 4 2018. [20] R. Krueger, Focus groups: a practical guide for applied re-
[3] B. Penzenstadler, L. Duboc, C. C. Venters, S. Betz, N. Seyff, search. Sage Publications, 1994.
K. Wnuk, R. Chitchyan, S. M. Easterbrook, and C. Becker, [21] B. H. Wixom and P. A. Todd, “A theoretical integration of user
“Software engineering for sustainability: Find the leverage satisfaction and technology acceptance,” Info. Sys. Research,
points!,” IEEE Software, vol. 35, pp. 22–33, July 2018. vol. 16, pp. 85–102, Mar. 2005.
[4] R. Chitchyan and et al, “Sustainability design in requirements [22] G. Assefa and B. Frostell, “Social sustainability and social
engineering: State of practice,” in ICSE Companion, pp. 533– acceptance in technology assessment: A case study of energy
542, 2016. technologies,” Technology in Society, vol. 29, no. 1, 2007.
[5] R. Nidumolu, C. Prahalad, and M. Rangaswami, “Why sustain- [23] Y. I. Ormeno, J. I. Panach, N. Condori-Fernndez, and O. Pastor,
ability is now the key driver of innovation,” IEEE Engineering “Towards a proposal to capture usability requirements through
Management Review, vol. 43, no. 2, pp. 85–91, 2015. guidelines,” in IEEE 7th International Conference on Research
[6] C. Calero, M. Á. Moraga, and M. F. Bertoa, “Towards a soft- Challenges in Information Science (RCIS), pp. 1–12, May 2013.
ware product sustainability model,” CoRR, vol. abs/1309.1640, [24] C. Becker, R. Chitchyan, L. Duboc, S. Easterbrook, B. Penzen-
2013. stadler, N. Seyff, and C. C. Venters, “Sustainability design and
[7] C. Venters and et. al, “The blind men and the elephant: Towards software: The karlskrona manifesto,” in Proceedings of the 37th
an empirical evaluation framework for software sustainability,” International Conference on Software Engineering - Volume 2,
Journal of Open Research Software, vol. 2, no. 1, 2014. ICSE ’15, (USA), pp. 467–476, IEEE Press, 2015.
[8] P. Lago, S. A. Koçak, I. Crnkovic, and B. Penzenstadler, [25] ISO/IEC, “ISO/IEC 25010 - Systems and software engineering
“Framing sustainability as a property of software quality,” - Systems and software Quality Requirements and Evaluation
Communications of the ACM, vol. 58, no. 10, pp. 70–78, 2015. (SQuaRE) - System and software quality models,” tech. rep.,
2010.
[9] N. Condori-Fernandez and P. Lago, “Characterizing the con-
tribution of quality requirements to software sustainability,” [26] C. Venters, C. Jay, L. Lau, M. K. Griffiths, V. Holmes, R. Ward,
Journal of systems and software, vol. 137, pp. 289–305, 2018. J. Austin, C. E. Dibsdale, and J. Xu, “Software sustainability:
The modern tower of babel,” in RE4SuSy: Third International
[10] B. Penzenstadler, “Infusing green: Requirements engineering Workshop on Requirements Engineering for Sustainable Sys-
for green in and through software systems,” in Proceedings of tems, 2014.
the Third International Workshop on Requirements Engineer-
ing for Sustainable Systems, RE4SuSy 2014, co-located with [27] P. Lago, N. Meyer, M. Morisio, H. A. Müller, and G. Scan-
22nd International Conference on Requirements Engineering, niello, “Leveraging energy efficiency to software users: sum-
Sweden, August 26, 2014., pp. 44–53, 2014. mary of the second GREENS workshop, at ICSE 2013,” ACM
SIGSOFT Software, 2014.
[11] N. Condori-Fernández and P. Lago, “The influence of green
strategies design onto quality requirements prioritization,” in [28] G. Brundtland, M. Khalid, S. Agnelli, S. Al-Athel, B. Chidzero,
International Working Conference on Requirements Engineer- L. Fadika, V. Hauff, I. Lang, M. Shijun, M. Morino de Botero,
ing: Foundation for Software Quality, pp. 189–205, 2018. M. Singh, and S. Okita, “Our common future (brundtland
report),” tech. rep., World Commission on Environment and
[12] R. Chitchyan, J. Noppen, and I. Groher, “What can software Development, 21 May 1987.
engineering do for sustainability: Case of software product
lines,” in International Workshop on Product Line Approaches [29] P. Lago, “Software and sustainability [inaugural lecture].”
in Software Engineering, pp. 11–14, May 2015. http://dare.ubvu.vu.nl, Jan. 2016.
[13] G. A. Garcı́a-Mireles, M. Á. Moraga, F. Garcı́a, C. Calero, and [30] C. Becker and et al, “Requirements: The key to sustainability,”
M. Piattini, “Interactions between environmental sustainability IEEE Software, vol. 33, no. 1, pp. 56–65, 2016.
goals and software product quality: A mapping study,” Infor- [31] N. Condori-Fernandez and P. Lago, “Towards a software
mation & Software Technology, vol. 95, pp. 108–129, 2018. sustainability-quality model: Insights from a multi-case study,”
[14] N. Condori Fernandez and P. Lago, A Sustainability-quality in International Conference on Research Challenges in Infor-
Model: (version 1.0). VU Technical Report, 11 2018. mation Science, IEEE, 2019.
[15] P. Lago, “Architecture design decision maps for software sus-
tainability,” in International Conference on Software Engineer-
ing, Software Engineering in Society Track (ICSE SEIS), 2019.
10