- In ISO/IEC JTC 1 SC 7/WG 6 (Software Product and System Quality), the authors have served as Conveners and have been working on the JTC 1/SC 7/WG 6 international projects on systems and software quality evaluation. In this paper, we first explain the significance and background of international standardization in this area. Second, the structure of the ISO/IEC 25000 SQuaRE (Systems and software Quality Requirements and Evaluation) series developed by SC 7/WG 6 and the outline of the international standards within the series are explained. Finally, the strategies adopted to drive this international standardization are described.
ISO/IEC JTC 1 SC 7/WG 6 (Software Product and System Quality) has promoted international standardization of software and software-intensive systems quality from the early stage of international standardization of software engineering by ISO/ TC 97: Computers and Information Processing (merged into ISO/IEC JTC 1).
Today, systems, used for such as internet shopping and supply chain management, are connected across national and corporate boundaries, and products for IoT are connected dynamically. Those systems and products are utilized in our daily life. Under the situation, it is extremely important to establish international standards for systems and software quality and promote to utilize them for ensuring the safety of the user, and for ensuring compatibility and interoperability of the systems.
In this paper, we explain the significance of standardization on system and software quality, the background of the ISO/IEC JTC 1 SC 7/WG 6, and the overview of the latest international standards. Then explain the strategies behind them. The strategy will be described in three aspects: organizational, operational, and technical.
II. SIGNIFICANCE AND BACKGROUND OF INTERNATIONAL STANDARDIZATION OF SYSTEMS AND SOFTWARE QUALITY
The 1st ISO/TC 97/SC 7 International Meeting was held in Paris, France in December 1974. The 2nd was held in April 1976 in West Berlin, West Germany at that time. M.Azuma, one of the authors, participated in this meeting for the first time as a representative of Japan.
The theme of SC 7 at that time was graphic symbols for information processing, but M. Azuma realized that it was too outdated in the period when software engineering had been
F. ISO/IEC 9126 and ISO/IEC 14598
The ISO/IEC 9126: Quality characteristics and guidelines for their use was published in 1991. Due to the success of the ISO/IEC 9126 software quality model, in order to make it easier to use, it was decided to enhance it with two series of international standards, i.e., ISO/IEC 9126 series (4 parts) regarding the quality models and measures of software products and ISO/IEC 14598 series (6 parts) regarding the quality evaluation of software products.
When the SC 7/WG 6 International Meeting was held in Kanazawa, Japan in November 1999, when both the ISO/IEC 9126 and 14598 series were mostly completed, Japan proposed to integrate both series and give it the name SQuaRE (Software Quality Requirements and Evaluation) series, which is easy to remember, and presented the architecture of the SQuaRE series. The new SQuaRE series was proposed and approved at the JTC 1/SC 7 International Meeting held in Madrid, Spain in May 2000, and numbers from ISO/IEC 25000 to 25099 were reserved for the SQuaRE series. Through the evolution from ISO/IEC 9126 to ISO/IEC 25000 series, the scope of the quality models has enlarged from software products to quality in use, systems, data, and IT services. Quality measures on them have also been defined.
The ISO/IEC 25000 SQuaRE series developed after that will be explained in the next clause.
This division provides guidance on how to utilize the SQuaRE series in the life cycle of systems and software products to control their quality. Terms and basic concepts used throughout the series are defined.
Currently, SQuaRE provides guidance for applying SQuaRE and guidance for promoting quantitative quality control in an organization.
This division defines multiple quality models that can be used properly according to the evaluation target and stage. The quality models decompose the concept of quality into subordinate concepts called quality (sub)characteristics and define each of them.
Currently, SQuaRE defines quality models that can be used for quality requirement definition and quality evaluation for three types of target entities, i.e., systems and software products, data and IT services. For systems and software products and IT services, models from the production point of view and user point of view are provided.
This division defines the measurement framework, quality measures and measurement methods used for measuring quality (sub)characteristics. It also defines the quality measure elements (QME) used for calculating the values of quality measures.
Currently, SQuaRE provides the quality measurement framework and defines quality measures and measurement methods corresponding to quality models of systems and software products, data and user perspectives, and QMEs that are used for calculating the values of quality measures of systems and software products. The quality measures and measurement methods of IT services will be provided as TS soon.
This division defines the framework of specifying quality requirements using quality models and quality measures.
Currently, SQuaRE defines the quality requirements framework covering system and software product quality requirements, data quality requirements, and IT service quality requirements.
This division defines a quality evaluation process using a quality model and quality measures.
Currently, SQuaRE defines quality evaluation framework and practical evaluation guidance for developers, acquires and independent evaluators, and provides a framework to store the evaluation knowledge.
This division provides complementary requirements and guidelines for utilizing SQuaRE effectively in a specific context.
Currently, SQuaRE provides requirements for independently evaluating Ready-to-Use Software Products (RUSP) and various formats for usability evaluation called CIF (Common Industry Format for usability).
Quality models for AI systems and cloud computing services will be provided in the future.
STANDARDIZATION
As mentioned above, we have been involved in the establishment of international standards for systems and software quality. As Conveners, we focus on fairness and consensus by the rules of international standardization (Directives), encourage the continuous and active participation of experts in each country, and introduce advanced technologies and concepts to establish practical and high-quality international standards. In the following, the matters that we have emphasized in the process will be summarized from the three aspects, i.e., organizational, operational, and technical, as strategies for international standardization.
Japan’s contribution to the international standards of systems and software quality is mentioned in clause II. Proposals from Japan, such as a new structure of a series of international standards and new work items with working drafts, depend on the research activities in Japan. We are highly involved in domestic activities. In this clause, an organization and projects to support international standardization, which was established and operated in collaboration with industry, academia and public institution, are introduced.
the Information Technology Standardization Research Center
(INSTAC) has been established at the Japanese Standards
Association (JSA) on behalf of the Agency of Industrial
Science and Technology, Ministry of International Trade and
Industry. INSTAC consisted of four Working Groups (WG)
corresponding to the activities of ISO/TC 97/SC 7 at that time
and ISO/IEC JTC 1/SC 7 after that. The results were
published every year as the "Survey and Research Report of
Standardization of Software Development and System
Documentation", and was used as an input for the work of SC
7 through the members who also serve as members of SC 7.
In particular, the "Software Quality Evaluation
Guidebook"[
2011, the Software Metrics Advanced Project was set up at
Mitsubishi Research Institute, commissioned by the Ministry
of Economy, Trade and Industry (METI). This project
consisted of two groups, process metrics and product metrics.
As to the product metrics, software metrics used in practice
and possibly to use are surveyed for various types of software
products. The results of this project were translated into
English and published on the WWW in March 2011[
Engineering: From 2015 to 2017, Waseda University was
commissioned by the Information-technology Promotion
Agency (IPA) for "Quantification of the actual software
products quality by measurement, evaluation and analysis,
and establishment of a comprehensive quality evaluation
framework" as a RISE project. It was an empirical study of
SQuaRE. In this study, we measured the quality of 21 actual
software products, such as packaged software and cloud
applications, based on SQuaRE quality measures, and
analyzed trends by quality characteristics and relationships
between quality characteristics. This result was published as
Quality Benchmark WSQB17: Waseda Software Quality
Benchmark [
important things in the operation of international standardization is the building of good relationships. International name recognition through presentations, lectures, and appointment of committee members at many international conferences are useful for building good relationships. These achievements have supported the expertise, gained the trust of the committee members of each country, and have helped to promote projects smoothly. In addition, good human relations with the Chairperson and Secretary of SC7, and delegations of each country are also indispensable for the smooth operation of the WG. It is also important to continuously participate in international meetings to deepen exchanges, to sincerely fulfill the roles and to gain trust.
duties: In international standardization, it is expected to undertake appropriate roles within SC and WG with the support of the National Bodies of the participating countries. However, the appointed members may not be necessarily in charge of establishing international standards within their organizations, and a mechanism for backing up the duties of their roles is required. In Japan, the Information Technology Standards Commission of Japan (ITSCJ), Information Processing Society of Japan (IPSJ) on behalf of the Japanese Industrial Standards Committee (JISC) has set up a mirror committee that equals the organization of WGs within SC 7. Twenty experts participate in the domestic WG6 to correspond to and support international standardization. In addition, the ITSCJ staff supports the operation of SCs and WGs, such as holding domestic meetings and inviting and operating international meetings. In the operation of WG6, the load is distributed among the members by forming small groups for each project and sharing the international and domestic works within the small groups.
3) Promotion of international standards: The established international standards are meaningless unless they are recognized and used. In WG 6, the following activities are being carried out in collaboration with international and domestic members in order to promote their practical use. a) Issuing books b) Submission of papers to Journal
symposiums, etc.
necessary)
As to d), after the publication of an International Standard of ISO/IEC 9126, 14598 and 25000 series, a committee is established within ITSCJ/IPSJ (Information Technology Commission of Japan, Information Processing Society of Japan) to translate it into Japanese and publish as Japan Industrial Standards (JIS). The committee is supported by METI (Ministry of Economy, Trade and Industry) and consists of Japan’s SC 7/WG 6 members and experts on JIS. Through these activities, the ISO/IEC 25000 SQuaRE series has become one of the most frequently referred international standards within SC 7.
C. Technical Strategies
establishment of international standards, in addition to ensuring the consistency between the standards established by the WG in charge, it is necessary to consider the consistency with the standards established by other related WGs. WG 6 is working on standardization of system and software quality requirements and evaluation, and the other WGs within SC 7 working on standardization of documentation, process management, architecture and so on. At the higher level, JTC 1, ISO, and IEC, there are SCs and WGs working on standardization on a wider variety of topics. When establishing standards, if the scope is exclusive for each WG and the methods, concepts, and terms specified there do not affect others, we can focus only on the consistency of the standards developed by the WG, but in reality, the area is overlapped, and consistency with the standards developed by other SCs and WGs may be required. On the other hand, excessive adherence to their consistency may impair the consistency of the standards developed by the WG. WG 6 has a policy of placing the highest priority on the consistency of the standards set by the WG. In SQuaRErelated projects, we clarify the priorities and promote international standardization in the order of consistency within WG 6, consistency with other standards in SC 7, and consistency with standards outside SC 7.
2) Emphasis on practicality: The effectiveness of international standards in practice is indispensable. People with various expertise and background participate in WG 6 from industry, academia and public institution. Members of the WG 6 change frequently. So, there are cases where research-based proposals that are not sufficiently substantiated and intuitive proposals that ignore the history of past discussions are made. For these proposals, we explain the differences between international standards and research papers, the background of the discussion, seek the understanding of the proposers, and maintain consensus among WG members. When proposing new work items from Japan or developing and proposing working drafts, we take in the results of practical application and opinions from industry and users. In WG 6, future watch type activities as the following have been performed for seeking new market and technology.
In this paper, we explained the background of the international standardization of quality requirements and evaluation of systems and software products that SC 7/WG 6 is working on, and the outline of the international standards, called SQuaREseries, established by the WG. He also described the organizational, operational, and technical strategies of the WG.
A major factor to consider as a future strategy is the spread of Web meetings. Currently, at international meetings in SC 7, it is obligatory to set up a Web meeting in parallel with a faceto-face meeting when the international meeting is held twice a year. In WG 6, two meetings in 2020 are held with fully virtual under the influence of coronavirus. With its widespread use, there are many points to be considered, such as the literacy required of participants in standardization activities, how to communicate, and how to proceed with collaborative work.