Approach to Develop Quality Model of Cloud Services HyunChong Kim Jae-Hyo Lee Software Quality Certification Team Software Quality Certification Team TTA(Telecommunications Technology Association) TTA(Telecommunications Technology Association) SeongNam, South Korea SeongNam, South Korea hchkim@tta.or.kr jaehyo@tta.or.kr Abstract— SQuaRE Series provides practical quality models characteristic is divided into sub-characteristics. A set of for system and software products. However, it does not fit quality characteristics is named as Quality Model. Figure 1 completely with aspects of new technologies. There are unique shows the structure of Quality Model. characteristics of cloud services, so we need special considerations when measuring and evaluating them. Therefore, a new project for standardization has started to develop a quality model and measures that guide the quality evaluation of cloud services. In this paper, we would like to share the approach we used to define the quality model for cloud services. Keywords—SQuaRE, cloud computing, standardization, SaaS, quality model I. INTRODUCTION ISO/IEC JTC1/SC7 is the international standardization organization responsible for standardizing software and systems engineering, which consists of 13 working groups and FIGURE 1. QUALITY MODEL STRUCTURE 5 ad-hoc groups. WG6, Working Group 6, is in charge of A set of quality measures is defined in ISO/IEC 2502n software product measurement and evaluation. The standards which can be used to evaluate quality characteristics[2]. developed by WG6 are named as SQuaRE (Software product ISO/IEC 2502n refers to 25021, 25022, 25023, 25024 and Quality Requirements and Evaluation). 25025. For example, sub-characteristic ‘functional SQuaRE Series provides practical quality models for correctness’ has a measure ‘functional correctness’. system and software products. However, it does not fit Measurement is an activity of determining a quality value completely with aspects of new technologies. There are against a software product based on quality model. After unique characteristics of cloud services, so we need special measurement, we can get objective values. considerations when measuring and evaluating them. Even though SC38, responsible for the standardization of cloud computing, has developed many standards for cloud computing, there is no standard for quality evaluation. It is the reason why SC7/WG6 started a new project to develop a quality model and measures that guide the quality evaluation of cloud services. Because it is the first time to apply the existing quality models to new technology, it is important to decide how to define the new model based on the existing models. It was also important to consider alignment with standards for cloud services. With these considerations, we proposed a FIGURE 2. RELATIONSHIP BETWEEN QUALITY MODEL AND methodology to defining a quality model for cloud services. MEASUREMENT In this paper, we would like to share the approach we has proposed to define the quality model for cloud services. Table Ⅰ shows the example of measures, defined in ISO/IEC 25023, which consists of a name, description, II. WHAT IS QUALITY AND QUALITY MODELS? measurement function and so on[3]. A. Definition and Concepts TABLE I. FUNCTIONAL CORRECTNESS MEASURE DEFINED IN ISO/IEC 25000 of SQuaRE Series provides quality ISO/IEC 25023 models as a defined set of quality characteristics. It provides a ID Name Description Measurement function good framework when specifying requirements and X = 1-A/B evaluating software quality. Quality characteristic is a What proportion of A = Number of functions FCr- Functional category of attributes relevant to software quality. functions provides that are incorrect 1-G Correctness B= Number of functions the correct results? considered Systems and software products have eight quality characteristics that are functional suitability, performance efficiency, compatibility, usability and so on[1]. Each quality Copyright © 2021 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). B. SC7/WG6 Standards • Step 5) Add new measures for the newly defined Table Ⅱ shows International Standards and Technical characteristics Specification developed by SC7/WG6. SC7/WG6 name these • Step 6) Provide the guideline for the model and standards as SQuaRE (Software product Quality measurement standards Requirements and Evaluation). There are five divisions that are 1) Product Quality General, 2) Quality Model, 3) Quality Measurement, 4) Quality Requirements and 5) Evaluation. Until now, four quality models have been developed. ISO/IEC 25010[1] defines two quality models of system & software products, ISO/IEC 25011[4] defines the IT service quality model and ISO/IEC 25012 [5] defines the data quality model. TABLE II. SC7/WG6 STANDARDS (NAMED SQUARE SERIES) Standard # Name of Standards Division ISO/IEC 25000 Guide to SQuaRE Product Quality ISO/IEC 25001 Planning and Management General ISO/IEC 25010 System and Software Quality Models Quality FIGURE 3. APPROACH TO DEVELOP THE QUALITY MODEL OF CLOUD ISO/IEC TS 25011 IT Service Quality Model SERVICES Model ISO/IEC 25012 Data Quality Model IV. QUALITY MODEL DEVELOPMENT OF CLOUD ISO/IEC 25020 Measurement Reference Model SERVICES ISO/IEC 25022 Measurement of Quality in Use In this chapter, we describe in detail the three steps to ISO/IEC 25023 Measurement of Sys. & SWP Quality Quality define the quality model. Measurement ISO/IEC 25024 Measurement of Data Quality A. (Step 1) Selecting Sub-characteristics from Existing ISO/IEC TS 25025 Measurement of IT Service Quality Quality Models for Cloud Services Provided by CSP (TS) 1) Application of Product Quality Model Quality ISO/IEC 25030 Quality Requirement Requirement Since CSP(Cloud Service Provider) provides a software, platform and infrastructure as a service, cloud services ISO/IEC 25040 Quality Evaluation Process provided by CSP can be evaluated by system and software ISO/IEC 25041 Evaluation Guide for Developers, Evaluation product quality model, called ‘product quality model’, defined Acquirers and Independent Evaluators in ISO/IEC 25010 [1]. ISO/IEC 25045 Evaluation Module for Recoverability As explained earlier, the product quality model is applicable to both computer systems and software products. The computer system includes computer hardware, software III. DEVELOPMENT APPROACH and data, not communication systems. With the cloud computing point of view, we can interpret the computer In SQuaRE Series, there are well-defined quality models system and software products as infrastructure, platform and for measuring and evaluating system & software products, IT application provided by CSP. For this reason, we can apply service, data, and so on. To define the quality model of cloud this model to cloud services. services, we can first use the existing models and then add new quality characteristics based on SC38 standards. 2) Application of IT Services Quality Model There are two types of IT services which are defined in Figure 3 shows the methodology to develop the quality ISO/IEC TS 25011 [4]: model of cloud services. To define the quality model, we propose to take the following steps for the quality model • Services completely automated provided by an IT development. system (type a) • Step 1) Select quality characteristics from the existing • Services provided by a human using an IT system (type quality models of SQuaRE Series b) • Step 2) Suggest new quality sub-characteristics based According to the types of IT services, cloud services on SC38 standards provided by CSP are highly related to ‘type a’ because they have the following characteristics: • Step 3) Classify the sub-characteristics into appropriate characteristics • CSC(Cloud Service Customer) provisions computing capabilities automatically or with minimal interaction After the quality model is defined, the next steps to with CSP develop quality measures and a guideline are as follows: • One or more capabilities offered via cloud computing • Step 4) Select appropriate quality measures from the invoked using a defined interface. In other words, the existing measurement standards capabilities are offered with minimal interaction As well as type a), we also should consider type b) in some reason why we should be concerned with the data quality for sense, because there are sub-roles of CSP, customer support PaaS and IaaS because they handle data intensively. For this and care representative that provide customer services to CSC. reason, we will apply the data quality model for PaaS and IaaS. For this reason, we can partially apply the IT service quality In case of SaaS, it is enough to apply the product quality model model. without the data quality model. Figure 6 and 7 show the suggested application scope of the quality models of SaaS, There are similar sub-characteristics between the product PaaS and IaaS. quality model and IT service quality model. For this case, we suppose that the product quality model takes precedent over the service model because CSP provides the services by using the existing functions rather than implementing new components. That is why the yellow circle is in front of the rectangle and triangle in Figure 4. Because we partially apply the IT service quality model, the partial blue rectangle is used for defining quality model of cloud services. Because the product and data quality model can be fully applied, a full circle and triangle are used in Figure 4. Figure 5 shows SQuaRE view of the quality model development approach. FIGURE 6. QUALITY MODEL OF SAAS BY CSP FIGURE 4. QUALITY MODEL DEVELOPMENT APPROACH OF CLOUD SERVICES BY CSP FIGURE 7. QUALITY MODEL OF PAAS AND IAAS BY CSP B. (Step 2) Adding New Sub-characteristics based on SC38 Standards For Step 2, we define new quality sub-characteristics by analyzing the following aspects of cloud computing that are specified in the standards of SC38 (See Figure 8). • Key Characteristics of Cloud Computing described in ISO/IEC 22123-1 [6] • Cross-Cutting Aspects of Cloud Computing described in ISO/IEC 17789 [7] • Cloud Service Level Agreement (SLA) described in FIGURE 5. QUALITY MODEL DEVELOPMENT APPROACH OF CLOUD ISO/IEC 19086-1 [8], 19086-2 [9], 19806-3 [10] and SERVICES BY CSP WITH SQUARE VIEW 19086-4 [11] • Interoperability and Portability described in ISO/IEC 3) Application of Data Quality Model 19941 [12] Although quality characteristics for data are implicitly connected to the product quality model, data quality model has • Data Flow, Data Categories and Data Use described been defined to complement the product quality model. in ISO/IEC 19944 [13] In case of system software such as operating systems, data For example, one of the key characteristics of cloud quality is more crucial than application software. This is the computing is ‘measured service’ because CSC pays for cloud services based on how much they use resources through services, the second step is the most important in defining a measured service. To provide measured service, the resource cloud quality model. For this reason, we need to conduct usage should be monitored, controlled, reported, and billed. further studies on which kinds of quality sub-characteristics To evaluate this feature, we suggest the measured services as should be included in the quality model of cloud services by the new sub-characteristics. analyzing the key aspects of cloud computing. REFERENCES [1] ISO/IEC 25010:2011 Systems and software engineering — Systems and software Quality Requirements and Evaluation (SQuaRE) — System and software quality models [2] ISO/IEC 25020:2019 Information Technology — Systems and Software Quality Tequirements and Evaluation (SQuaRE)— Quality Measurement Framework [3] ISO/IEC 25023:2016 Systems and software engineering — Systems and software Quality Requirements and Evaluation (SQuaRE) — Measurement of system and software product quality [4] ISO/IEC TS 25011:2017 Information technology — Systems and software Quality Requirements and Evaluation (SQuaRE) — Service quality models [5] ISO/IEC 25012:2008 Software engineering — Software product Quality Requirements and Evaluation (SQuaRE) — Data quality FIGURE 8. STEP 2 TO DEFINE THE QUALITY MODEL FOR CLOUD SERVICES model [6] ISO/IEC 22123-1 Information technology — Cloud computing — Part C. (Step 3) Sub-characteristics Classification to 1: Vocabulary Appropriate Characteristics [7] ISO/IEC 17789:2014 Information technology — Cloud computing — In the previous chapter, we suggested the full set of quality Reference architecture sub-characteristics for CSP. In Step3, we classify the newly [8] ISO/IEC 19086-1:2016 Information technology — Cloud computing defined sub-characteristics into appropriate characteristics. If — Service level agreement (SLA) framework — Part 1: Overview and there is no appropriate characteristic, we propose the new one. concepts Figure 9 explains the key idea of sub-characteristics [9] ISO/IEC 19086-2:2018 Cloud computing — Service level agreement classification into the appropriate characteristics. After taking (SLA) framework — Part 2: Metric model Step 3, we will have a quality model of cloud services [10] ISO/IEC 19086-3:2017 Information technology — Cloud computing provided by CSP. — Service level agreement (SLA) framework — Part 3: Core conformance requirements [11] ISO/IEC 19086-4:2019 Cloud computing — Service level agreement (SLA) framework — Part 4: Components of security and of protection of PII [12] ISO/IEC 19941:2017 Information Technology — Cloud Computing — interoperability and portability [13] ISO/IEC 19944:2020 Cloud computing and distributed platforms ─ Data flow, data categories and data use — Part 1: Fundamentals FIGURE 9. STEP 3 TO DEFINE THE QUALITY MODEL FOR CLOUD SERVICES V. CONCLUSION AND FUTURE WORK The SQuaRE standards provide a basis for evaluating systems and software products. However, since it does not fit well with the new technology. For this reason, this paper proposed an approach to develop the quality model of cloud services. In this paper, we proposed three steps to develop a quality model of cloud services. As the first step, we selected the appropriate quality sub-characteristics from the existing quality models with consideration of the characteristics of cloud services. The second step reflects the characteristics of cloud services into the quality model, so we suggested how to define new quality sub-characteristics for cloud computing. Last step classifies the quality sub-characteristics into characteristics. To apply the existing quality model to cloud