Evaluation of Software Product Quality Attributes and Environmental Attributes using ANP Decision Framework Sedef Akınlı Koçak Gülfem Işıklar Alptekin Ayşe Başar Bener Data Science Laboratory Computer Engineering Data Science Laboratory Environmental Applied Science Galatasaray University Mechanical and Industrial Ryerson University Istanbul, Turkey Engineering, Ryerson University Toronto, Canada gisiklar@gsu.edu.tr Toronto, Canada Sedef.akinlikocak@ryerson.ca Ayse.bener@ryerson.ca requirements (green requirements) to the system. Those green Abstract—Green software development is a relatively new requirements generally create conflicts on both functional research area within green IT. Software development industry requirements and quality attributes. Interdependencies has started getting pressure from regulators to consider green between environmental concerns and quality attributes are software development. As a result, green attributes of software typically hidden compared to the dependencies between products are gaining importance as quality attributes. In this functional requirements and quality attributes. Hence, in order study, we evaluate environmental sustainability and software quality criteria using a well-known multi criteria decision making to develop environmentally sustainable software products, we approach: Analytical Network Process (ANP). The aim is to need to address these conflicts and prioritize concerns and determine the relationships among quality and environmental requirements. A multi-criteria decision making (MCDM) attributes and relative priorities of attributes. The results are approach is suitable for this aim since it incorporates multiple presented as a guide for green software developers. The priority objectives for prioritizing conflicts [6]. MCDM approaches are weights of attributes may be used to analyze trade-off between known for their flexibility in handling complex information, conflicting product quality and environmental requirements. and their ability to construct values during priority-setting in Index Terms— Green Software, Software Quality, trade-off analysis. Environmental Sustainability, Analytic Network Process In our earlier work [7], we built a framework using ANP which is a widely used MCDM approach to prioritize quality I. INTRODUCTION and environmental criteria in a general sense. The work in this In software development, comprehensive specification and paper extends our previous framework by defining sub-criteria evaluation of software product quality is a key factor in of quality and environmental criteria. The main contribution of ensuring desired level of quality. A desired level of quality for this work is that it expands the framework that enables decision software may be achieved by defining appropriate quality makers determining the importance of each requirement when characteristics, taking into account the purpose of usage of the deciding on the technical specifications of the software. A software product. trade-off analysis may also be applied during requirement Similarly, green information and communication analysis step where making decisions regarding sustainability technologies (ICT) and green software have become important affects both software product development and software system research areas that aim to achieve environmentally sustainable itself. computing systems. Designing green and sustainable software Another key consideration for delivering business value with that also satisfies customer needs in new software products environmental responsiveness is deciding what requirements to and/ or legacy software are complex tasks. Developing green develop and how to develop. At this point, trade-offs among software necessitates identification of the development different requirements and stakeholders emerge. Trade-offs are requirements as well as quality metrics. There have been inevitable and need to be balance wisely. Our research seeks to various software quality models in the literature. These models analyze the various trade-off possibilities. propose sets of metrics to evaluate both product and process This paper is organized as follows: Section II summarizes quality [1, 2, 3, 4]. the concepts of software quality models and evaluations, Quality requirements (aka quality attributes) and customer requirement prioritization, software quality and environmental requirements (aka functional requirements) sometimes may attributes. This section also presents the related literature. In conflict. In these cases, practitioners make trade-off decisions Section III, the methodology used in the evaluation framework or prioritization to solve the conflict. However, in most cases is briefly explained. The analysis, the results and the discussion functional requirements and quality attributes depend on each with trade-off analysis are presented in Section IV. Section V other, thereby complicating the trade-off decisions [5]. In gives conclusions and future work. addition, sustainability concerns induce environmental II. RELATED WORK AND BACKGROUND Requirement prioritization is a process of managing the Researchers have been studying sustainability of software relative importance and determination of different requirements from different perspectives: hardware energy efficiency in within the limited resources. It is a challenging activity to find terms of power consumption; optimization of algorithms and / the right balance among competing quality requirements. In or software architecture; effective and efficient usage of this paper, ANP, a well-known MCDM approach, is used to software engineering practices and tools as defined in ISO/IEC prioritize the criteria. It improves the quality of decisions by system and software engineering standard series [8, 9, 10]. providing information on trade-offs, and it increases confidence in decisions and provide insights into the criteria and the A. Software Quality Models alternatives. In decision support systems studies, Analytical Software quality is defined as a “set of features and Hierarchy Process (AHP), hybrid ANP/AHP method, fuzzy characteristics of product or service that bears on its ability to AHP [11], AHP-goal model, binary search tree creation, satisfy the stated or implied needs” [8]. Software development greedy-type algorithms [12] are the most commonly used projects need to be completed within certain constraints such as methods. Most of the above mentioned tools and methods time, budget, and resources. Therefore, managers need to make recognize the AHP as a reference method. However, AHP is trade-off decisions to determine how much functionality/ or difficult to use with large sets of requirements. In order to solve which features will be included in each release. In software this problem, a case-based ranking method [13] is introduced engineering literature, various software quality models have for decision support system based on a machine learning been proposed. Some of the most standard and well-known approach. Another technique is called quality function quality models are McCall’s model [1], Boehm’s model [2], deployment (QFD) which helps the organization from the early Bowen’s model [3], Dromey’s model [4], ISO/EC 9126-1 [8], stages of production process (i.e. designing phase) to reach the and ISO/EC 25010 [10] models. The considered quality criteria customer’s satisfaction with proper technical requirements. in each model is summarized in Table 1. The common criteria QFD has been adopted as a software quality function found in the majority of these models are: efficiency, deployment (SQFD) [14] and utilized to plan and design new reliability, usability, functionality, maintainability, and software or improved software. Although the number of portability. research on applying SQFD to software industry is limited compared to manufacturing industry, the use of QFD in TABLE I. QUALITY MODELS IN RELATED LITERATURE software engineering has gained more attention in recent years. Criteria/Goals McCall Boehm Bowen Dromey ISO-9126 ISO-25010 C. Software Quality Attributes and Sustainability [1] [2] [3] [4] [8] [10] Maintainability + + + + + Various systems (e.g. energy systems, management Flexibility + + systems, and computer systems) take sustainability as an Testability + + objective for reaching a high quality level. Penzenstadler, Correctness + + Mahaux, and Salinesi [15] stated that current discussions on the Efficiency + + + + + sustainability requirements foster on how to define, measure Reliability + + + + + + and assess sustainability as quality attribute of software. A first Integrity + + quality model for green and sustainable software was Usability + + + + + developed by Kern, Dick, Naumann, Guldner, and Johann [16]. Portability + + + + + + It refers to quality factors from ISO /IEC 25000, and it is based Reusability + + + on the directly and indirectly related criteria of software. It Interoperability + + gives an overview of potential aspects which may be taken as Understandability + sustainability criteria and metrics for software products. Calero Functionality + + + and Bertoa [17] considered sustainability as a new factor that Modifiability + affects software product and process quality. In their work, a Performance + new quality model (ISO 2510+S) is presented based on Supportability ISO/25010. The authors differentiated the quality factors in Security + respect to the sustainability impact and described related and Compatibility + unrelated sub-characteristics. All of these studies discuss the relation of the software quality with the sustainability in a B. Decision Making in Software Requirements general manner. They point out that the product, as well as the Software development is a design intensive process. Every quality in use, needs to be considered when evaluating the person involved in this process has to make a number of sustainability of the software. So far, to the best of our decisions, each of them with several possible choices. Most of knowledge, there is no empirical study that uses the quality the time, decisions are made according to personal knowledge attributes with sustainability attributes. and experience. But, as software development projects get D. Environmental Criteria and Green Metrics complex and diverse, decision making becomes a group activity where people need to use decision tools and techniques With the purpose of clarifying different ways that ICT can to communicate and coordinate their decisions. contribute to the reduction of CO2 emissions and sustainable development, three levels of effects of ICT have been identified [18, 19]: direct, indirect and systematic. In terms of TABLE II. GREEN METRICS CLASSIFICATION identifying environmental criteria, in this paper, we are IT resource interested in direct effects. Direct effects refer to those caused Lifecycle Energy Impact Organizational usage by software product, i.e. the resource consumption when using - CPU usage - Lifecycle cost - Consumables - Laws and the applications. Environmental performance of the software - Memory - Process - System regulations systems can be evaluated with environmental criteria and usage engineering power/energy - Resource efforts green metrics. They are also necessary for practitioners to - Storage - Quality of usage - Greenhouse gas usage service - Greenhouse credits develop, deploy and control the system sustainability. Here, we - I/O usage gas (e.g. CO2 - Green solutions address this point by analyzing green metrics as a criteria set. emissions) Those criteria may be used in setting up environmental objectives and evaluating the sustainability of software while GPIs measure the system greenness with respect to products. In order to refine such a group of criteria, we energy consumption. Their approach focuses on GPIs, in reviewed recent green metrics studies. Kansal and Zhao [20] particular, usage and energy related indicators. Although most presented an energy profiling metric for applications at design of them may represent different system layers, their proposed phase. The energy consumed by an application is divided into framework turns indicators through normalization functions two parts: the performance required by the application and the such that they can be compared and aggregated within system resources used. Their application energy profiling meaningful clusters. approach considers how energy consumption is spanned across In our study, we focus on the criteria that are directly related the involved resources such as CPU, disk, memory and to the quality and environmental attributes. Hence, we network interfaces during different states. Lago and Jansen [21] aggregate two sub-criteria sets for our study using Kipp, Jiang, focused on process, service and people aspects and proposed a and Fugini [24] and Ferreira, Pernici, and Plebani [25] framework which makes environmental strategies and green classifications: resource consumption that have the higher indicators available as-a-service. Their adaptation strategies are influence over energy consumption and energy impact which is based on one single metric for each. The metrics are not closely related to the resource usage. The cluster criteria, sub- analyzed together and their dependencies are not considered. criteria and their indicators are listed in Section IV.B. However, in our proposed approach, we analyze priorities based on the dependencies of metrics using ANP approach. III. ANALYTIC NETWORK PROCESS (ANP) In a recent systematic literature review, Bozzelli, Gu, and The ANP generalizes a widely used multi-criteria decision Lago [22] identified that most of the measures were defined to making tool, the AHP, by replacing hierarchies with networks. measure energy consumption, either directly or as a The AHP is a well-known technique that decomposes a performance indicator. Following that study, Agarwal, Nath, problem into several levels in such a way that they form a and Chowdhur [23] indicated that software performance hierarchy [6]. ANP uses a network structure with depends on the basic computing resources like the CPU and the interrelationships among the decision levels and attributes [26]. memory. Software is required for faster CPUs and increased A relative ratio scale of each element measurement is derived memory. In this situation, sustainability is affected from the from pairwise comparisons of the elements in a level of the increasing resource consumption. Therefore, we concluded that hierarchy with respect to an element of the preceding level. we need to evaluate software products sustainability in terms of ANP provides a general approach to deal with decisions resource usage. considering the dependence of higher level elements to lower According to Kipp, Jiang, and Fugini [24], green metrics are level elements and the dependence of the elements within a the key drivers to identify the “greenness” of an IT application level. In general the ANP consists of three stages: and to indicate the energy consumption, energy efficiency, and 1. Defining the goal, criteria, sub-criteria, energy saving possibilities. They classify four clusters of 2. Defining the interdependencies and the network, and metrics: a) IT resource usage that is related to energy 3. Calculation of the priorities of the elements [26]. consumption, b) lifecycle metrics that monitor energy The relative importance value according to pairwise consumption and metrics that develop energy aware comparisons between criteria and sub-criteria are performed indicators, c) energy impact metrics that are related to the using a scale of 1–9, where a score of 1 indicates equal lifecycle impact on the environment (i.e. the electricity, the importance between the two elements and 9 represents the power supply, the consumed material, and the CO2 emissions), extreme importance of one element compared to the other one. and d) organizational metrics that consider the assessment of The relations are calculated as in Equation.1 additional costs due to energy-related initiatives. The following table (Table II) summarizes the main categories of metrics used for each cluster [24]. a ji  1 (1) aij Using the same classification, Ferreira, Pernici, and Plebani [25] recently have presented two different types of indicators for different system layers: Key Performance Indicators where aij denotes the importance of the ith element compared to (KPIs) and Green Performance Indicators (GPIs). KPIs the jth element, and aii=1 are preserved in the pairwise measure system attributes that are based on quality models, comparison matrix to improve the consistency of the judgments. The ANP requires that the consistencies of pairwise comparison matrices are checked to prove the consistency of Each step is presented in more detail in the following sub- comparisons. A zero value would indicate perfect consistency sections. whereas larger values indicating increasing levels of B. Determination of Criteria and Sub-Criteria inconsistency. Saaty [26] has indicated that the consistency should be about 10% or less to be acceptable. If it exceeds the Quality criteria (functionality, usability, efficiency, 10% level, value judgment may need to be revised. reliability) and their related sub-criteria were adopted from After calculating the importance aij, the composite weights common software product criteria in the literature. are extracted considering the interrelationships between criteria Environmental criteria were adopted from IT resource usage and sub-criteria [27]. We utilized ANP approach to define and energy impact categories (see sub-section II.D) [24]. priority weights of the quality and environmental criteria and Table III describes the criteria and the selected sub-criteria. their sub-criteria. This approach best suits in our aim since it Figure 1 depicts our decision framework. Here metrics and identifies the interdependencies between criteria and their sub- indicators are used interchangeably. criteria in the network structure. For environmental dimension of sustainability, the classification is important since all the interdependencies IV. EVALUATION FRAMEWORK related to both quality and environment are elicited for the trade-off analysis. Similar classification may also be used to A. Integrating Environmental Requirements in Decision identify other aspects of sustainability dimensions that Making correspond to quality requirements. Users as well as other stakeholders, such as project managers, regulators, other decision makers, recognize the TABLE III. SOFTWARE QUALITY AND ENVIRONMENTAL CRITERIA limited resources of the system and introduce green requirements to the development process. In addition to Software Quality Criteria (Q) [8] quality requirements, environmental concerns possess multiple A set of attributes that bear on the existence of a set of functions and their viewpoints and conflicts. They must be identified and Q1 Functionality specified properties; the functions are eventually resolved. The value here is not only identifying the those that satisfy stated or implied needs. conflicts but also eliciting the opportunities resulting from the A set of attributes that bear on the effort trade-offs. Q2 Usability needed for use, and on the individual assessment of such use, by a stated or Resource concerns, efficiency and time constraints have implied set of users. direct effects on software development life cycle. Integrating A set of attributes that bear on the green requirements to the development process creates Q3 Reliability capability of software to maintain its level diversity in understandings, source of knowledge, and of performance under stated conditions for a stated period of time. opportunities based on interdependencies rather than conflicts A set of attributes that bear on the between requirements. Resource usage, for example, directly relationship between the level of leads to changes in efficiency and reliability, which affects the Q4 Efficiency performance of the software and the quality. While resource usage affects functionality of the amount of resources used, under stated conditions. product, it also brings the opportunity to save energy [28, 29]. Environmental Sustainability Criteria (E) [24, 25] As a result, presence of green requirements in the process The energy consumption by a software of developing the decision framework provides an opportunity relates both to the nature of the application E1 Resource usage to explore and construct different development outcomes and to the system configuration of the run regarding different goals. time environment Metrics relate to the impact of the The objective of our proposed framework is to determine application on the environment, the priority weights of software quality and environmental E2 Energy impact considering electricity, power supply, characteristics. Those weights will help to identify technical consumed material, and emissions requirements and their correlations that will be useful in design and development processes of green software. Calculation of the metrics used in the framework are not C. Questionnaire and Pairwise Comparisons detailed in this paper since the main focus is on the relative Our proposed approach has two major criteria in the first importance and priorities of the criteria on their cluster. The level, six sub-criteria in the second level and twenty one sub- decision process addresses the problem of selecting the most criteria in the third level. In our previous study [7], we important criteria that a software developer needs to focus in calculated the priority weights of two criteria (quality and the design process. The process can be divided into three main environment) and related sub-criteria (Qi, Ei). steps: We prepared a questionnaire for experts in order to find out 1. Identifying the criteria and sub-criteria and their the weights for criteria and interactions among all the second interdependencies. and third level indicators. Hence, we extended the framework 2. Making pairwise comparisons using questionnaire. by integrating third level indicators. The questionnaire was 3. Obtaining the interdependent priorities of the criteria filled out by the same expert group as in our first study. Experts and sub-criteria. are from four different software development companies. They are software experts with an experience of 4 to 20 years in the field. Khorramshahgol and Moustakis [30] pointed out that best personal interviews. At the beginning of the questionnaire, the suited number of experts as an interviewee is in general 5–15 meaning of the criteria and comparison of criteria were persons. provided to advance understanding of the contents in the In order to avoid any vagueness or hard readability that questionnaire by the experts. The geometric mean of 10 may affect the answers given by the experts, and facilitate the questionnaires is used to form the output of the pairwise understanding about criteria and the ANP model, we conducted comparisons [27]. Fig. 1. Research Framework D. Analysis of the interdependencies between the criteria and TABLE IV. PAIRWISE COMPARISON OF ENVIRONMENTAL (E) SUB- CRITERIA sub-criteria In this step of the methodology, we applied the ANP E1 E2 E1 1 2 approach. We used the Super Decisions1 software tool in order E2 1/2 1 to calculate the results of final priorities of the sub-criteria (Figure 1). The Super Decisions is a simple and easy-to-use TABLE V. PAIRWISE COMPARISON OF QUALITY (Q) SUB-CRITERIA software for constructing decision models with dependence and feedback. The consistency indexes of all these matrices were Q1 Q2 Q3 Q4 Q1 1 2 3 1/3 found to be smaller than 0.10, which is in the acceptable range. Q2 1/2 1 1/2 1/3 Small consistency indicates the consistency of experts’ Q3 1/3 2 1 1/4 evaluations. The pairwise comparisons enabled us to retrieve Q4 3 3 4 1 relative weights of criteria and sub-criteria and indicators. TABLE VI. PAIRWISE COMPARISON OF FUNCTIONALITY (Q1) INDICATORS E. Results and Discussion UNDER THE QUALITY Each expert has to decide which one of the two given Q11 Q12 Q13 Q14 criteria is more important in developing a green and sustainable Q11 1 4 2 2 software product, along with its relative importance degree. Q12 1/4 1 1/2 1/2 They defined the degree of importance using the 1-9 scale. (For Q13 1/2 2 1 1/2 Q14 1/2 2 2 1 instance, 9 means that the first criterion is extremely important compared to the second one). In order to determine aggregate individual judgments, we took the geometric average of the TABLE VII. PAIRWISE COMPARISON OF USABILITY (Q2) INDICATORS UNDER THE QUALITY responses. Table IV through Table XI represent the geometric means of pairwise comparisons between environmental Q21 Q22 Q23 Q24 Q21 1 2 1 3 sustainability criteria and quality criteria. Q22 1/2 1 2 2 Q23 1 1/2 1 3 Q24 1/3 1/2 1/3 1 1 Super Decisions (ver. 2.2.6) [Open source Software]. Available from: http://www.superdecisions.com TABLE VIII. PAIRWISE COMPARISON OF RELIABILITY (Q3) INDICATORS The results indicate that, among the two environmental UNDER THE QUALITY criteria, the resource usage criterion has the highest priority Q31 Q32 Q33 Q34 with 66%. This shows that it has a high impact on green Q31 1 1 2 3 software development. Among the four quality related sub- Q32 1 1 1 1 criteria, the reliability has the highest priority with 43%, Q33 1/2 1 1 2 Q34 1/3 1 1/2 1 followed by the efficiency and the functionality with 24%, and 23%, respectively. TABLE IX. PAIRWISE COMPARISON OF THE EFFICIENCY (Q4) INDICATORS The reliability of a system is a measure of its ability to UNDER THE QUALITY provide a failure-free operation. This also means the actual Q41 Q42 usage time of the product by user. Therefore, the reliability is Q41 1 3 correlated with the behavior of software with respect to time, Q42 1/3 1 and it is one of the sub-criteria of the efficiency. The TABLE X. PAIRWISE COMPARISON OF THE RESOURCE USAGE (E1) efficiency is the second important quality criteria, with 28% of INDICATORS UNDER THE ENVIRONMENT weight, followed by usability and functionality. Software E11 E12 E13 E14 efficiency considers the amount of information technology E11 1 2 2 4 resources that are used efficiently in terms of energy. E12 1/2 1 2 3 Undoubtedly, the priority weights may depend on the software E13 1/2 1/2 1 3 type. E14 1/4 1/3 1/3 1 The results related to environmental criteria show that CPU usage has the highest priority with 43% followed by I/O TABLE XI. PAIRWISE COMPARISON OF THE ENERGY IMPACT (E2) usage and memory usage with 24% and storage usage with INDICATORS UNDER THE ENVIRONMENT 9%. This demonstrates that the most important energy related E21 E22 E23 resource is the CPU. The resource usage has a direct influence E21 1 2 1 E22 1/2 1 2 on the energy consumption sub-criteria, which has the higher E23 1 1/2 1 relative weight with 41% in the energy impact criteria followed by CO2 emission with 33%. These comparison values were used as input to the ANP Figure 2 shows the comparison of results of our findings methodology, and we calculated the priorities which are in this paper with our previous study [7]. The results reveal summarized in Table XII. As the last step of the proposed that the priority ranking of the criteria is different. In our procedure, the final priorities of the each criteria, sub-criteria previous work, the efficiency had the highest priority (34%) and indicators were calculated. followed by reliability (24%), functionality (23%) and usability (19%). However, in this work, the reliability of the TABLE XII. SUB-CITERIA AND INDICATOR LEVEL WEIGHTS product has the highest weight (43%) followed by efficiency (28%), usability (14%) and functionality (13%). The reason is Relative Quality Criteria Weight Indicators Weight that the integration of more detailed sub-criteria and indicators enables DMs to better analyze interdependencies. Therefore, Availability 0.37 the final priority ranking may change if we ignore the Fault Tolerance 0.24 interactions of detailed sub–criteria and indicators. Reliability 0.43 Recoverability 0.23 Maturity 0.15 Accuracy 0.44 Suitability 0.22 Functionality 0.13 Security 0.22 Interoperability 0.11 Operability 0.36 Understandability 0.28 Usability 0.14 Learnability 0.26 Attractiveness 0.11 Time Behaviour 0.75 Efficiency 0.28 Resource 0.25 Utilization Environmental Relative Weight Indicators Criteria Weight CPU Usage 0.43 I/O Usage 0.24 Resource Usage 0.66 Fig. 2. Comparison of priority weight results with our previous work [7] Memory Usage 0.24 Storage Usage 0.09 Energy Consumption 0.41 Energy Impact 0.33 CO2 Emission 0.33 Green Energy Usage 0.26 F. Assessing the Criteria and Trade-off Analysis indicators. In the process of creating the ANP decision The sub-criteria describe the entire system with a given framework, the discussion with experts was made before the goal and are, therefore, usable and widely understood relationships among levels and criteria are given. Therefore, indicators of quality and green attributes of software. Criteria the ANP structure in this research is reliable and valid. with higher influence to the system’s goal get higher weights. Using our approach and framework, decision makers may For instance, considering the energy consumption, criteria determine requirements’ priorities and use the framework as related to the CPU power consumption have higher importance the basis to see the potential outcomes and impacts of the than I/O. Hence, we can conclude that CPU related events criteria. Additionally, this approach offers an opportunity to represent higher risk for both quality and environment. properly shape software development projects from the The set of quality and environmental criteria are highly beginning to the maintenance phase in a strategic way. related to the time behaviour that has high impact on resource Next steps will be constructing a QFD model for new usage, energy consumption and efficiency. The results indicate green software product development, by using the ANP two major trade-offs: First one is between reliability, which is weights obtained from this study. We plan to integrate a goal threatened by performance degradation, system behavior and programming technique to QFD in order to consider resource functionality loss, and limited resource usage. The second one and budget limitations. is between energy impact which is threatened by energy REFERENCES consumption and related CO2 emissions, and efficiency of the product which is threatened by time behaviour. Although [1] J. A. McCall, P. K. Richards, and G. F. Walters. Factors in resource usage has high priority, resource utilization does not software quality, General Electric, National Technical Information Service, 1977. show the same behaviour as quality criteria. The analysis reveals that future action would be converting [2] B. W. Boehm, J. R. Brownand, and J. R. Kaspar, Characteristics reliability, efficiency and resource usage trade-offs into the of Software Quality, TRW Series of Software Technology, Amsterdam, North Holland, 1978. opportunities for saving energy. [3] T .P. Bowen, G. B. Wigle, and J. T. Tsai. “Specification of G. Threats to Validity software quality attributes,” Technical Report RADC-TR-85-37, The overall objective of the study is to evaluate the Rome Air Development Center, 1985. priorities of the criteria to achieve green software. We [4] R. G. Dromey, “A model for software product introduced a prioritization method to gain a better quality,” Software Engineering, IEEE Transaction, vol.21, no.2, 1995, pp. 146-162. understanding of interdependencies of the quality and environmental criteria in green software development. [5] E. E. Karsak, S. Sozer and S. E. Alptekin “Product planning in Selection of the criteria may introduce bias. We chose the quality function deployment using a combined analytic network process and goal programming approach,”, Computers & quality criteria from well-known and mostly used quality Industrial Engineering, vol. 44, 2002, pp. 171–190. models. We selected the environmental criteria from the IT related environmental studies. Environmental applications on [6] T. L. Saaty, The Analytic Hierarchy Process, McGraw-Hill, IT systems are fairly new research area. Therefore, the New York, NY, 1980. environmental criteria might be different according to the [7] S. Akınlı Koçak, G. Gonzales Calienes, G. Işıklar Alptekin, and application area. The number of questionnaires filled by the A. Başar Bener, “Requirements prioritization framework for experts may be another threat to validity. Since a continuous developing green and sustainable software using ANP - based decision making,” In Proceedings of the Environmental nature of the study, we needed to interview the same DMs in Informatics Conference, 2013 (Hamburg, Germany, September order to be consistent and be able to make comparisons. 2-4, 2013). V. CONCLUSION AND FUTURE WORK [8] ISO/IEC 9126-1. Information technology - Software quality characteristics and metrics - Part 1: Quality characteristics and Most software development companies need to adapt their sub-characteristics, ISO, Int. Electrotechnical Commission, development processes to meet new software business Geneva, 2001. standards. One of these standards is the development of [9] ISO/IEC IS 12207. Systems and software engineering- Software environmentally friendly products. In many cases, these life cycle processes, International Organization for objectives are achieved at a cost of reducing or increasing the Standardization, Geneva, 2008. consumption of various resources within the organization. As [10] ISO/IEC, IS 25010. Systems and software engineering - Systems a result, the software development industry needs making and software Quality Requirements and Evaluation (SQuaRE) - certain trade-offs between the demands of end-users and the System and software quality models, Geneva, 2011. requirements for environmental sustainability. This work is an [11] Z Sener and E. E. Karsak. “A fuzzy regression and optimization initial attempt to prioritize quality and environmental approach for setting target levels in software quality function attributes, and further to build trade-off models. We made use deployment,” Software Quality Journal, 2010, vol.18, pp.323– of ANP, a well-known MCDM technique, for dealing with 339. conflicting criteria. [12] G., Ruhe, A., Eberlein, and D. Pfahl, “Quantitative Win-win: A The proposed ANP framework is subject to the judgment of New Method for Decision Support in Requirements experts for the comparison among the criteria, sub-criteria and Negotiation,” In proceeding of the 14th Int’l Conf. Software Eng. and Knowledge Eng., (Ischia, Italy), 2002, pp.159-166. [13] P. Avesani, C. Bazzanella, A. Perini, and A. Susi. “Supporting [22] P. Bozzelli, Q. Gu, and P. Lago. “A systematic literature review the Requirements Prioritization Process. A Machine Learning on green software metrics,” Technical Report: VU University Approach,” In proceeding of 16th Int’l Conf. Software Eng. and Amsterdam, 2013, Available at: Knowledge Eng. (Maurer F. and Ruhe, G., Eds, Banff, Canada), http://www.sis.uta.fi/~pt/TIEA5_Thesis_Course/Session_10_20 2004, pp.306-311. 13_02_18/SLR_GreenMetrics.pdf [14] S. Haag, M. Raja, and L. L. Schkade. “Quality function [23] S. Agarwal, A. Nath, and D. Chowdhury, “Sustainable deployment usage in software development,” Communications Approaches and Good Practices in Green Software of the ACM, 1996, vol.39, no.1, pp.41-49. Engineering,” International Journal of Research and Reviews in [15] B. Penzenstadler, M. Mahaux, and C. Salinesi. “Requirements Computer Science (IJRRCS), 2012, vol. 3, no.1. pp. 1425-1428. engineering for sustainable systems, In proceeding of the [24] A.Kipp, T. Jiang, and M. Fugini, “Green metrics for energy- International Working Conference on Requirements aware IT systems,” IEEE Computer Society, 2011, vol.42, Engineering: Foundation for Software Quality. (Essen, pp.241-248. Germany), 2012, vol.52, pp.8-13. [25] A. M. Ferreira, B. Pernici, and P. Plebani. “Green performance [16] E. Kern, M. Dick, S, Naumann, A, Guldner, and T. Johann, indicators aggregation through composed weighting “Green Software and Green Software Engineering – Definitions, system,” ICT as Key Technology against Global Warming. Measurements, and Quality Aspects,” In proceeding of the First Springer Berlin Heidelberg, 2012, pp.79-93. International Conference on ICR for Sustainability (Zurich, [26] T. L. Saaty, Decision making with dependence and feedback: Switzerland, 14-16 February 2013), 2013, pp.87-94. The analytic network process. RWS Publications, 1996, [17] C. Calero and M. F. Bertoa. “25010+S: A software quality Pittsburgh, PA. model with sustainable characteristics. Sustainability as an [27] T. L. Saaty and L. G. Vargas “Diagnosis with dependent element of software quality,” In Proceeding of the GIBSE 2013, symptoms: Bayes theorem and the analytic hierarchy process" (Fukuoka, Japan), 2013. Operations Research, 1998, vol.46, no.4, pp.491–502. [18] Organization for Economic Co-operation and Development [28] S. Akınlı Koçak, A. Miransky, G. Işıklar Alptekin, A. Başar (OECD), 2010, OECD Greener and Smarter: ICTs, the Bener, E. Cialini, “The impact of improving software environment and climate change 2010. OECD Pub. Paris, functionality on environmental sustainability,” In: Proceedings France. of the First International Conference on ICT for Sustainability [19] L. M., Hilty, P. Arnfalk, L. Erdmann, J. Goodman, M. Lehmann, (ICT4S, Zurich, Switzerland), 2013, pp.104-110. and P. A. Wäger, “The relevance of information and [29] A. Miransky, S. Akınlı Koçak, E. Cialini, and A. Başar Bener, communication technologies for environmental sustainability–A “A. Save energy with the DB2 10.1 for Linux, UNIX, and prospective simulation study,” Environmental Modelling & Windows data compression feature,” IBM DeveloperWoks, Software, 2006, vol.21, no.1, pp.1618-1629. Technical Library, 2013, Available at: [20] A. Kansal, F. Zhao, “Fine-grained energy profiling for power- http://www.ibm.com/developerworks/data/library/techarticle/dm aware application design,” SIGMETRICS Performance -1302db2compression/ Evaluation Review, 2008, vol.36, no.2, pp.26–31. [30] Khorramshahgol, R., and Moustakis, V. S. “Delphic hierarchy [21] P. Lago, T. Jansen, “Creating Environmental Awareness in process (DHP): A methodology for priority setting derived from Service Oriented Software Engineering,” In: Maximilien, E.M., the Delphi method and analytical hierarchy process,” European Rossi, G., Yuan, S.-T., Ludwig, H., Fantinato, M. (eds.) ICSOC Journal of Operational Research, 1988, vol.37, no.3, pp.347-354. 2010. LNCS, Springer, Heidelberg, 2011, vol. 6568, pp. 181– 186.