Establishing International Standards for Systems and Software Quality Requirements and Evaluation Toshihiro Komiyama Motoei Azuma Software Engineering Division Waseda University NEC Corporation Tokyo, Japan Tokyo, Japan azumam@waseda.jp t-komiyama@nec.com Abstract— In ISO/IEC JTC 1 SC 7/WG 6 (Software Product getting popular and proposed the Chairperson to change the and System Quality), the authors have served as Conveners and title and scope of SC 7 to Software Engineering and approved. have been working on the JTC 1/SC 7/WG 6 international projects on systems and software quality evaluation. In this C. Software Quality Model Project Started paper, we first explain the significance and background of It began with the SC 7/WG 3 Munich Meeting in February international standardization in this area. Second, the structure 1985. At the meeting, Japan proposed the creation of an of the ISO/IEC 25000 SQuaRE (Systems and software Quality international standard for software quality models. During the Requirements and Evaluation) series developed by SC 7/WG 6 meeting, based on a proposal from Japan, a quality model was and the outline of the international standards within the series created from scratch using the KJ method, which is a problem- are explained. Finally, the strategies adopted to drive this solving technique developed by Jiro Kawakita. The international standardization are described. participants reached a consensus earlier than expected, and it Keywords—international standard, ISO/IEC 25000 series, was agreed to initiate an international standard of software SQuaRE, systems and software quality quality model based on it. After that, SC 7 moved to ISO/IEC JTC 1 (Information technology) established in 1987. I. INTRODUCTION D. Establishment of SC 7/WG 6 ISO/IEC JTC 1 SC 7/WG 6 (Software Product and System At the SC 7/WG 3 International Meeting in Budapest, Quality) has promoted international standardization of Hungary in November 1989, it was proposed and approved to software and software-intensive systems quality from the divide the SC 7/WG 3 into three Sub-Groups in order to work early stage of international standardization of software efficiently. M. Azuma was appointed as the leader of SG 2 in engineering by ISO/ TC 97: Computers and Information charge of the quality model. Furthermore, at the SC 7 meeting Processing (merged into ISO/IEC JTC 1). held in Stockholm, Sweden in June 1991, it was decided that Today, systems, used for such as internet shopping and each SG of SC 7/WG 3 would be a WG, SC 7/WG 3/SG 2be supply chain management, are connected across national and SC 7/WG 6, and M. Azuma be the Convener. corporate boundaries, and products for IoT are connected E. Operation of SC 7/WG 6 International Meetings dynamically. Those systems and products are utilized in our daily life. Under the situation, it is extremely important to The 1st SC 7 / WG 6 International Meeting was held in establish international standards for systems and software Turin, Italy, in November 1991, courtesy of Italian Telecom. quality and promote to utilize them for ensuring the safety of Since then, international conferences have been held twice a the user, and for ensuring compatibility and interoperability of year, with the participating countries rotating around. In 2015, the systems. T. Komiyama, another one of the authors, took over the Convener from M. Azuma. In this paper, we explain the significance of standardization on system and software quality, the F. ISO/IEC 9126 and ISO/IEC 14598 background of the ISO/IEC JTC 1 SC 7/WG 6, and the The ISO/IEC 9126: Quality characteristics and guidelines overview of the latest international standards. Then explain for their use was published in 1991. Due to the success of the the strategies behind them. The strategy will be described in ISO/IEC 9126 software quality model, in order to make it three aspects: organizational, operational, and technical. easier to use, it was decided to enhance it with two series of international standards, i.e., ISO/IEC 9126 series (4 parts) II. SIGNIFICANCE AND BACKGROUND OF INTERNATIONAL regarding the quality models and measures of software STANDARDIZATION OF SYSTEMS AND SOFTWARE QUALITY products and ISO/IEC 14598 series (6 parts) regarding the A. Participation in ISO/TC 97/SC 7 quality evaluation of software products. The 1st ISO/TC 97/SC 7 International Meeting was held G. ISO/IEC 25000 SQuaRE Series in Paris, France in December 1974. The 2nd was held in April When the SC 7/WG 6 International Meeting was held in 1976 in West Berlin, West Germany at that time. M.Azuma, Kanazawa, Japan in November 1999, when both the ISO/IEC one of the authors, participated in this meeting for the first 9126 and 14598 series were mostly completed, Japan time as a representative of Japan. proposed to integrate both series and give it the name SQuaRE B. From Graphic Symbols to Software Engineering (Software Quality Requirements and Evaluation) series, which is easy to remember, and presented the architecture of The theme of SC 7 at that time was graphic symbols for the SQuaRE series. The new SQuaRE series was proposed and information processing, but M. Azuma realized that it was too approved at the JTC 1/SC 7 International Meeting held in outdated in the period when software engineering had been Madrid, Spain in May 2000, and numbers from ISO/IEC Copyright © 2020 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). 25000 to 25099 were reserved for the SQuaRE series. software products, data and user perspectives, and QMEs that Through the evolution from ISO/IEC 9126 to ISO/IEC 25000 are used for calculating the values of quality measures of series, the scope of the quality models has enlarged from systems and software products. The quality measures and software products to quality in use, systems, data, and IT measurement methods of IT services will be provided as TS services. Quality measures on them have also been defined. soon. The ISO/IEC 25000 SQuaRE series developed after that D. ISO/IEC 2503n - Quality Requirements Division will be explained in the next clause. This division defines the framework of specifying quality requirements using quality models and quality measures. Currently, SQuaRE defines the quality requirements framework covering system and software product quality requirements, data quality requirements, and IT service quality requirements. E. ISO/IEC 2504n - Quality Evaluation Division This division defines a quality evaluation process using a quality model and quality measures. Currently, SQuaRE defines quality evaluation framework and practical evaluation guidance for developers, acquires and independent evaluators, and provides a framework to store the evaluation knowledge. F. ISO/IEC 25050 25099-Extension Division This division provides complementary requirements and Fig. 1. The ISO/IEC 25000 SQuaRE Architecture guidelines for utilizing SQuaRE effectively in a specific context. III. OVERVIEW OF ISO/IEC 25000 SQUARE SERIES Currently, SQuaRE provides requirements for ISO / IEC 25000 SQuaRE series consists of six divisions[1, independently evaluating Ready-to-Use Software Products 2] As shown in Fig. 1. The outline of each division is as (RUSP) and various formats for usability evaluation called follows. CIF (Common Industry Format for usability). A. ISO/IEC 2500n - Quality Management Division Quality models for AI systems and cloud computing This division provides guidance on how to utilize the services will be provided in the future. SQuaRE series in the life cycle of systems and software IV. STRATEGIES AND INITIATIVES TO PROMOTE products to control their quality. Terms and basic concepts STANDARDIZATION used throughout the series are defined. As mentioned above, we have been involved in the Currently, SQuaRE provides guidance for applying establishment of international standards for systems and SQuaRE and guidance for promoting quantitative quality software quality. As Conveners, we focus on fairness and control in an organization. consensus by the rules of international standardization B. ISO/IEC 2501n - Quality Model Division (Directives), encourage the continuous and active participation of experts in each country, and introduce This division defines multiple quality models that can be advanced technologies and concepts to establish practical and used properly according to the evaluation target and stage. The high-quality international standards. In the following, the quality models decompose the concept of quality into matters that we have emphasized in the process will be subordinate concepts called quality (sub)characteristics and summarized from the three aspects, i.e., organizational, define each of them. operational, and technical, as strategies for international Currently, SQuaRE defines quality models that can be standardization. used for quality requirement definition and quality evaluation A. Organizational Strategies for three types of target entities, i.e., systems and software products, data and IT services. For systems and software Japan’s contribution to the international standards of products and IT services, models from the production point of systems and software quality is mentioned in clause II. view and user point of view are provided. Proposals from Japan, such as a new structure of a series of international standards and new work items with working C. ISO/IEC 2502n - Quality Measurement Division drafts, depend on the research activities in Japan. We are This division defines the measurement framework, quality highly involved in domestic activities. In this clause, an measures and measurement methods used for measuring organization and projects to support international quality (sub)characteristics. It also defines the quality measure standardization, which was established and operated in elements (QME) used for calculating the values of quality collaboration with industry, academia and public institution, measures. are introduced. Currently, SQuaRE provides the quality measurement 1) Creation and achievements of INSTAC: Since 1986, framework and defines quality measures and measurement the Information Technology Standardization Research Center methods corresponding to quality models of systems and (INSTAC) has been established at the Japanese Standards Association (JSA) on behalf of the Agency of Industrial meetings to deepen exchanges, to sincerely fulfill the roles Science and Technology, Ministry of International Trade and and to gain trust. Industry. INSTAC consisted of four Working Groups (WG) 2) Undertaking responsibilities and supporting the corresponding to the activities of ISO/TC 97/SC 7 at that time duties: In international standardization, it is expected to and ISO/IEC JTC 1/SC 7 after that. The results were undertake appropriate roles within SC and WG with the published every year as the "Survey and Research Report of support of the National Bodies of the participating countries. Standardization of Software Development and System However, the appointed members may not be necessarily in Documentation", and was used as an input for the work of SC charge of establishing international standards within their 7 through the members who also serve as members of SC 7. organizations, and a mechanism for backing up the duties of In particular, the "Software Quality Evaluation their roles is required. In Japan, the Information Technology Guidebook"[3] written based on the results of INSTAC/WG Standards Commission of Japan (ITSCJ), Information 4 became the basis of the ISO/IEC 9126 and 14598 series, Processing Society of Japan (IPSJ) on behalf of the Japanese and the ISO/IEC 25000 SQuaRE series. Activities of Industrial Standards Committee (JISC) has set up a mirror INSTAC continued until it was abolished in 2010. Since the committee that equals the organization of WGs within SC 7. results of INSTAC were directly reflected in these series of Twenty experts participate in the domestic WG6 to international standards and made a great contribution, its correspond to and support international standardization. In abolition is regrettable. addition, the ITSCJ staff supports the operation of SCs and 2) Software Metrics Advanced Project: From 2009 to WGs, such as holding domestic meetings and inviting and 2011, the Software Metrics Advanced Project was set up at operating international meetings. In the operation of WG6, Mitsubishi Research Institute, commissioned by the Ministry the load is distributed among the members by forming small of Economy, Trade and Industry (METI). This project groups for each project and sharing the international and consisted of two groups, process metrics and product metrics. domestic works within the small groups. As to the product metrics, software metrics used in practice 3) Promotion of international standards: The established and possibly to use are surveyed for various types of software international standards are meaningless unless they are products. The results of this project were translated into recognized and used. In WG 6, the following activities are English and published on the WWW in March 2011[4]. They being carried out in collaboration with international and were used for standardization of quality measurement in the domestic members in order to promote their practical use. SQuaRE series as practical and useful information. a) Issuing books 3) RISE: Research Initiative on Advanced Software b) Submission of papers to Journal Engineering: From 2015 to 2017, Waseda University was c) Lectures at academic societies, seminars, commissioned by the Information-technology Promotion symposiums, etc. Agency (IPA) for "Quantification of the actual software d) Issuance as domestic standards (with translation, if products quality by measurement, evaluation and analysis, necessary) and establishment of a comprehensive quality evaluation e) Support for establishment and operation of the framework" as a RISE project. It was an empirical study of certification scheme SQuaRE. In this study, we measured the quality of 21 actual software products, such as packaged software and cloud As to d), after the publication of an International Standard of applications, based on SQuaRE quality measures, and ISO/IEC 9126, 14598 and 25000 series, a committee is analyzed trends by quality characteristics and relationships established within ITSCJ/IPSJ (Information Technology between quality characteristics. This result was published as Commission of Japan, Information Processing Society of Quality Benchmark WSQB17: Waseda Software Quality Japan) to translate it into Japanese and publish as Japan Benchmark [5, 6]. Data collection and analysis continue and Industrial Standards (JIS). The committee is supported by the research results will be utilized for SQuaRE in the future., METI (Ministry of Economy, Trade and Industry) and consists of Japan’s SC 7/WG 6 members and experts on JIS. B. Operational Strategies Through these activities, the ISO/IEC 25000 SQuaRE series 1) Formation of personal connections: One of the most has become one of the most frequently referred international important things in the operation of international standards within SC 7. standardization is the building of good relationships. International name recognition through presentations, C. Technical Strategies lectures, and appointment of committee members at many 1) Maintaining consistency and integrity: In the international conferences are useful for building good establishment of international standards, in addition to relationships. These achievements have supported the ensuring the consistency between the standards established expertise, gained the trust of the committee members of each by the WG in charge, it is necessary to consider the country, and have helped to promote projects smoothly. In consistency with the standards established by other related addition, good human relations with the Chairperson and WGs. WG 6 is working on standardization of system and Secretary of SC7, and delegations of each country are also software quality requirements and evaluation, and the other indispensable for the smooth operation of the WG. It is also WGs within SC 7 working on standardization of important to continuously participate in international documentation, process management, architecture and so on. At the higher level, JTC 1, ISO, and IEC, there are SCs and WGs working on standardization on a wider variety of topics. c) Obtaining a higher view from JTC 1 focus like future When establishing standards, if the scope is exclusive for watch each WG and the methods, concepts, and terms specified V. CONCLUSIONS there do not affect others, we can focus only on the consistency of the standards developed by the WG, but in In this paper, we explained the background of the reality, the area is overlapped, and consistency with the international standardization of quality requirements and standards developed by other SCs and WGs may be required. evaluation of systems and software products that SC 7/WG 6 is working on, and the outline of the international standards, On the other hand, excessive adherence to their consistency called SQuaREseries, established by the WG. He also may impair the consistency of the standards developed by the described the organizational, operational, and technical WG. WG 6 has a policy of placing the highest priority on the strategies of the WG. consistency of the standards set by the WG. In SQuaRE- related projects, we clarify the priorities and promote A major factor to consider as a future strategy is the spread of Web meetings. Currently, at international meetings in SC 7, international standardization in the order of consistency it is obligatory to set up a Web meeting in parallel with a face- within WG 6, consistency with other standards in SC 7, and to-face meeting when the international meeting is held twice consistency with standards outside SC 7. a year. In WG 6, two meetings in 2020 are held with fully 2) Emphasis on practicality: The effectiveness of virtual under the influence of coronavirus. With its widespread international standards in practice is indispensable. People use, there are many points to be considered, such as the with various expertise and background participate in WG 6 literacy required of participants in standardization activities, from industry, academia and public institution. Members of how to communicate, and how to proceed with collaborative the WG 6 change frequently. So, there are cases where work. research-based proposals that are not sufficiently REFERENCES substantiated and intuitive proposals that ignore the history of [1] ISO/IEC 25000: Systems and software engineering — Systems and past discussions are made. For these proposals, we explain software Quality Requirements and Evaluation (SQuaRE) — Guide to the differences between international standards and research SQuaRE, 2014 papers, the background of the discussion, seek the [2] Komiyama, T.: Usability of Software-Intensive Systems from understanding of the proposers, and maintain consensus Developers’ Point of View - Current Status and Future Perspectives of International Standardization of Usability Evaluation, HCI among WG members. When proposing new work items from International 2020, July 2020 Japan or developing and proposing working drafts, we take [3] Azuma, M., et al: Software Quality Evaluation Guidebook, Japan in the results of practical application and opinions from Standards Association, 1994 industry and users. In WG 6, future watch type activities as [4] Ministry of Economy, Trade and Industry: Software Metrics Advanced the following have been performed for seeking new market Project, http://www.meti.go.jp/policy/it_policy/softseibi/ and technology. [5] Tsuda, N., et al: WSQF: comprehensive software quality evaluation framework and benchmark based on SQuaRE, Proceedings of the 41st a) Setting up WG 6-led Ad-hoc group International Conference on Software Engineering, pp 312–321, 2019 [6] WSQB17: Waseda Software Quality Benchmark, b) Utilizing SC7 level topics future watch(SC 7/AG 5) http://www.washi.cs.waseda.ac.jp/wsqb/