=Paper=
{{Paper
|id=Vol-2800/paper-02
|storemode=property
|title=Establishing International Standards for Systems and Software Quality Requirements and Evaluation
|pdfUrl=https://ceur-ws.org/Vol-2800/paper-02.pdf
|volume=Vol-2800
|authors=Toshihiro Komiyama,Motoei Azuma
|dblpUrl=https://dblp.org/rec/conf/apsec/KomiyamaA20
}}
==Establishing International Standards for Systems and Software Quality Requirements and Evaluation==
https://ceur-ws.org/Vol-2800/paper-02.pdf
Establishing International Standards for Systems
and Software Quality Requirements and Evaluation
Toshihiro Komiyama Motoei Azuma
Software Engineering Division Waseda University
NEC Corporation Tokyo, Japan
Tokyo, Japan azumam@waseda.jp
t-komiyama@nec.com
Abstract— In ISO/IEC JTC 1 SC 7/WG 6 (Software Product getting popular and proposed the Chairperson to change the
and System Quality), the authors have served as Conveners and title and scope of SC 7 to Software Engineering and approved.
have been working on the JTC 1/SC 7/WG 6 international
projects on systems and software quality evaluation. In this C. Software Quality Model Project Started
paper, we first explain the significance and background of It began with the SC 7/WG 3 Munich Meeting in February
international standardization in this area. Second, the structure 1985. At the meeting, Japan proposed the creation of an
of the ISO/IEC 25000 SQuaRE (Systems and software Quality international standard for software quality models. During the
Requirements and Evaluation) series developed by SC 7/WG 6 meeting, based on a proposal from Japan, a quality model was
and the outline of the international standards within the series created from scratch using the KJ method, which is a problem-
are explained. Finally, the strategies adopted to drive this solving technique developed by Jiro Kawakita. The
international standardization are described.
participants reached a consensus earlier than expected, and it
Keywords—international standard, ISO/IEC 25000 series,
was agreed to initiate an international standard of software
SQuaRE, systems and software quality quality model based on it. After that, SC 7 moved to ISO/IEC
JTC 1 (Information technology) established in 1987.
I. INTRODUCTION
D. Establishment of SC 7/WG 6
ISO/IEC JTC 1 SC 7/WG 6 (Software Product and System At the SC 7/WG 3 International Meeting in Budapest,
Quality) has promoted international standardization of Hungary in November 1989, it was proposed and approved to
software and software-intensive systems quality from the divide the SC 7/WG 3 into three Sub-Groups in order to work
early stage of international standardization of software efficiently. M. Azuma was appointed as the leader of SG 2 in
engineering by ISO/ TC 97: Computers and Information charge of the quality model. Furthermore, at the SC 7 meeting
Processing (merged into ISO/IEC JTC 1). held in Stockholm, Sweden in June 1991, it was decided that
Today, systems, used for such as internet shopping and each SG of SC 7/WG 3 would be a WG, SC 7/WG 3/SG 2be
supply chain management, are connected across national and SC 7/WG 6, and M. Azuma be the Convener.
corporate boundaries, and products for IoT are connected E. Operation of SC 7/WG 6 International Meetings
dynamically. Those systems and products are utilized in our
daily life. Under the situation, it is extremely important to The 1st SC 7 / WG 6 International Meeting was held in
establish international standards for systems and software Turin, Italy, in November 1991, courtesy of Italian Telecom.
quality and promote to utilize them for ensuring the safety of Since then, international conferences have been held twice a
the user, and for ensuring compatibility and interoperability of year, with the participating countries rotating around. In 2015,
the systems. T. Komiyama, another one of the authors, took over the
Convener from M. Azuma.
In this paper, we explain the significance of
standardization on system and software quality, the F. ISO/IEC 9126 and ISO/IEC 14598
background of the ISO/IEC JTC 1 SC 7/WG 6, and the The ISO/IEC 9126: Quality characteristics and guidelines
overview of the latest international standards. Then explain for their use was published in 1991. Due to the success of the
the strategies behind them. The strategy will be described in ISO/IEC 9126 software quality model, in order to make it
three aspects: organizational, operational, and technical. easier to use, it was decided to enhance it with two series of
international standards, i.e., ISO/IEC 9126 series (4 parts)
II. SIGNIFICANCE AND BACKGROUND OF INTERNATIONAL regarding the quality models and measures of software
STANDARDIZATION OF SYSTEMS AND SOFTWARE QUALITY products and ISO/IEC 14598 series (6 parts) regarding the
A. Participation in ISO/TC 97/SC 7 quality evaluation of software products.
The 1st ISO/TC 97/SC 7 International Meeting was held G. ISO/IEC 25000 SQuaRE Series
in Paris, France in December 1974. The 2nd was held in April When the SC 7/WG 6 International Meeting was held in
1976 in West Berlin, West Germany at that time. M.Azuma, Kanazawa, Japan in November 1999, when both the ISO/IEC
one of the authors, participated in this meeting for the first 9126 and 14598 series were mostly completed, Japan
time as a representative of Japan. proposed to integrate both series and give it the name SQuaRE
B. From Graphic Symbols to Software Engineering (Software Quality Requirements and Evaluation) series,
which is easy to remember, and presented the architecture of
The theme of SC 7 at that time was graphic symbols for the SQuaRE series. The new SQuaRE series was proposed and
information processing, but M. Azuma realized that it was too approved at the JTC 1/SC 7 International Meeting held in
outdated in the period when software engineering had been Madrid, Spain in May 2000, and numbers from ISO/IEC
Copyright © 2020 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
25000 to 25099 were reserved for the SQuaRE series. software products, data and user perspectives, and QMEs that
Through the evolution from ISO/IEC 9126 to ISO/IEC 25000 are used for calculating the values of quality measures of
series, the scope of the quality models has enlarged from systems and software products. The quality measures and
software products to quality in use, systems, data, and IT measurement methods of IT services will be provided as TS
services. Quality measures on them have also been defined. soon.
The ISO/IEC 25000 SQuaRE series developed after that D. ISO/IEC 2503n - Quality Requirements Division
will be explained in the next clause. This division defines the framework of specifying quality
requirements using quality models and quality measures.
Currently, SQuaRE defines the quality requirements
framework covering system and software product quality
requirements, data quality requirements, and IT service
quality requirements.
E. ISO/IEC 2504n - Quality Evaluation Division
This division defines a quality evaluation process using a
quality model and quality measures.
Currently, SQuaRE defines quality evaluation framework
and practical evaluation guidance for developers, acquires and
independent evaluators, and provides a framework to store the
evaluation knowledge.
F. ISO/IEC 25050 25099-Extension Division
This division provides complementary requirements and
Fig. 1. The ISO/IEC 25000 SQuaRE Architecture guidelines for utilizing SQuaRE effectively in a specific
context.
III. OVERVIEW OF ISO/IEC 25000 SQUARE SERIES Currently, SQuaRE provides requirements for
ISO / IEC 25000 SQuaRE series consists of six divisions[1, independently evaluating Ready-to-Use Software Products
2] As shown in Fig. 1. The outline of each division is as (RUSP) and various formats for usability evaluation called
follows. CIF (Common Industry Format for usability).
A. ISO/IEC 2500n - Quality Management Division Quality models for AI systems and cloud computing
This division provides guidance on how to utilize the services will be provided in the future.
SQuaRE series in the life cycle of systems and software IV. STRATEGIES AND INITIATIVES TO PROMOTE
products to control their quality. Terms and basic concepts STANDARDIZATION
used throughout the series are defined.
As mentioned above, we have been involved in the
Currently, SQuaRE provides guidance for applying establishment of international standards for systems and
SQuaRE and guidance for promoting quantitative quality software quality. As Conveners, we focus on fairness and
control in an organization. consensus by the rules of international standardization
B. ISO/IEC 2501n - Quality Model Division (Directives), encourage the continuous and active
participation of experts in each country, and introduce
This division defines multiple quality models that can be advanced technologies and concepts to establish practical and
used properly according to the evaluation target and stage. The high-quality international standards. In the following, the
quality models decompose the concept of quality into matters that we have emphasized in the process will be
subordinate concepts called quality (sub)characteristics and summarized from the three aspects, i.e., organizational,
define each of them. operational, and technical, as strategies for international
Currently, SQuaRE defines quality models that can be standardization.
used for quality requirement definition and quality evaluation A. Organizational Strategies
for three types of target entities, i.e., systems and software
products, data and IT services. For systems and software Japan’s contribution to the international standards of
products and IT services, models from the production point of systems and software quality is mentioned in clause II.
view and user point of view are provided. Proposals from Japan, such as a new structure of a series of
international standards and new work items with working
C. ISO/IEC 2502n - Quality Measurement Division drafts, depend on the research activities in Japan. We are
This division defines the measurement framework, quality highly involved in domestic activities. In this clause, an
measures and measurement methods used for measuring organization and projects to support international
quality (sub)characteristics. It also defines the quality measure standardization, which was established and operated in
elements (QME) used for calculating the values of quality collaboration with industry, academia and public institution,
measures. are introduced.
Currently, SQuaRE provides the quality measurement 1) Creation and achievements of INSTAC: Since 1986,
framework and defines quality measures and measurement the Information Technology Standardization Research Center
methods corresponding to quality models of systems and (INSTAC) has been established at the Japanese Standards
Association (JSA) on behalf of the Agency of Industrial meetings to deepen exchanges, to sincerely fulfill the roles
Science and Technology, Ministry of International Trade and and to gain trust.
Industry. INSTAC consisted of four Working Groups (WG) 2) Undertaking responsibilities and supporting the
corresponding to the activities of ISO/TC 97/SC 7 at that time duties: In international standardization, it is expected to
and ISO/IEC JTC 1/SC 7 after that. The results were undertake appropriate roles within SC and WG with the
published every year as the "Survey and Research Report of support of the National Bodies of the participating countries.
Standardization of Software Development and System However, the appointed members may not be necessarily in
Documentation", and was used as an input for the work of SC charge of establishing international standards within their
7 through the members who also serve as members of SC 7. organizations, and a mechanism for backing up the duties of
In particular, the "Software Quality Evaluation their roles is required. In Japan, the Information Technology
Guidebook"[3] written based on the results of INSTAC/WG Standards Commission of Japan (ITSCJ), Information
4 became the basis of the ISO/IEC 9126 and 14598 series, Processing Society of Japan (IPSJ) on behalf of the Japanese
and the ISO/IEC 25000 SQuaRE series. Activities of Industrial Standards Committee (JISC) has set up a mirror
INSTAC continued until it was abolished in 2010. Since the committee that equals the organization of WGs within SC 7.
results of INSTAC were directly reflected in these series of Twenty experts participate in the domestic WG6 to
international standards and made a great contribution, its correspond to and support international standardization. In
abolition is regrettable. addition, the ITSCJ staff supports the operation of SCs and
2) Software Metrics Advanced Project: From 2009 to WGs, such as holding domestic meetings and inviting and
2011, the Software Metrics Advanced Project was set up at operating international meetings. In the operation of WG6,
Mitsubishi Research Institute, commissioned by the Ministry the load is distributed among the members by forming small
of Economy, Trade and Industry (METI). This project groups for each project and sharing the international and
consisted of two groups, process metrics and product metrics. domestic works within the small groups.
As to the product metrics, software metrics used in practice 3) Promotion of international standards: The established
and possibly to use are surveyed for various types of software international standards are meaningless unless they are
products. The results of this project were translated into recognized and used. In WG 6, the following activities are
English and published on the WWW in March 2011[4]. They being carried out in collaboration with international and
were used for standardization of quality measurement in the domestic members in order to promote their practical use.
SQuaRE series as practical and useful information. a) Issuing books
3) RISE: Research Initiative on Advanced Software b) Submission of papers to Journal
Engineering: From 2015 to 2017, Waseda University was c) Lectures at academic societies, seminars,
commissioned by the Information-technology Promotion symposiums, etc.
Agency (IPA) for "Quantification of the actual software
d) Issuance as domestic standards (with translation, if
products quality by measurement, evaluation and analysis, necessary)
and establishment of a comprehensive quality evaluation
e) Support for establishment and operation of the
framework" as a RISE project. It was an empirical study of
certification scheme
SQuaRE. In this study, we measured the quality of 21 actual
software products, such as packaged software and cloud As to d), after the publication of an International Standard of
applications, based on SQuaRE quality measures, and ISO/IEC 9126, 14598 and 25000 series, a committee is
analyzed trends by quality characteristics and relationships established within ITSCJ/IPSJ (Information Technology
between quality characteristics. This result was published as Commission of Japan, Information Processing Society of
Quality Benchmark WSQB17: Waseda Software Quality Japan) to translate it into Japanese and publish as Japan
Benchmark [5, 6]. Data collection and analysis continue and Industrial Standards (JIS). The committee is supported by
the research results will be utilized for SQuaRE in the future., METI (Ministry of Economy, Trade and Industry) and
consists of Japan’s SC 7/WG 6 members and experts on JIS.
B. Operational Strategies Through these activities, the ISO/IEC 25000 SQuaRE series
1) Formation of personal connections: One of the most has become one of the most frequently referred international
important things in the operation of international standards within SC 7.
standardization is the building of good relationships.
International name recognition through presentations, C. Technical Strategies
lectures, and appointment of committee members at many 1) Maintaining consistency and integrity: In the
international conferences are useful for building good establishment of international standards, in addition to
relationships. These achievements have supported the ensuring the consistency between the standards established
expertise, gained the trust of the committee members of each by the WG in charge, it is necessary to consider the
country, and have helped to promote projects smoothly. In consistency with the standards established by other related
addition, good human relations with the Chairperson and WGs. WG 6 is working on standardization of system and
Secretary of SC7, and delegations of each country are also software quality requirements and evaluation, and the other
indispensable for the smooth operation of the WG. It is also WGs within SC 7 working on standardization of
important to continuously participate in international documentation, process management, architecture and so on.
At the higher level, JTC 1, ISO, and IEC, there are SCs and
WGs working on standardization on a wider variety of topics. c) Obtaining a higher view from JTC 1 focus like future
When establishing standards, if the scope is exclusive for watch
each WG and the methods, concepts, and terms specified
V. CONCLUSIONS
there do not affect others, we can focus only on the
consistency of the standards developed by the WG, but in In this paper, we explained the background of the
reality, the area is overlapped, and consistency with the international standardization of quality requirements and
standards developed by other SCs and WGs may be required. evaluation of systems and software products that SC 7/WG 6
is working on, and the outline of the international standards,
On the other hand, excessive adherence to their consistency
called SQuaREseries, established by the WG. He also
may impair the consistency of the standards developed by the described the organizational, operational, and technical
WG. WG 6 has a policy of placing the highest priority on the strategies of the WG.
consistency of the standards set by the WG. In SQuaRE-
related projects, we clarify the priorities and promote A major factor to consider as a future strategy is the spread
of Web meetings. Currently, at international meetings in SC 7,
international standardization in the order of consistency
it is obligatory to set up a Web meeting in parallel with a face-
within WG 6, consistency with other standards in SC 7, and to-face meeting when the international meeting is held twice
consistency with standards outside SC 7. a year. In WG 6, two meetings in 2020 are held with fully
2) Emphasis on practicality: The effectiveness of virtual under the influence of coronavirus. With its widespread
international standards in practice is indispensable. People use, there are many points to be considered, such as the
with various expertise and background participate in WG 6 literacy required of participants in standardization activities,
from industry, academia and public institution. Members of how to communicate, and how to proceed with collaborative
the WG 6 change frequently. So, there are cases where work.
research-based proposals that are not sufficiently
REFERENCES
substantiated and intuitive proposals that ignore the history of
[1] ISO/IEC 25000: Systems and software engineering — Systems and
past discussions are made. For these proposals, we explain software Quality Requirements and Evaluation (SQuaRE) — Guide to
the differences between international standards and research SQuaRE, 2014
papers, the background of the discussion, seek the [2] Komiyama, T.: Usability of Software-Intensive Systems from
understanding of the proposers, and maintain consensus Developers’ Point of View - Current Status and Future Perspectives of
International Standardization of Usability Evaluation, HCI
among WG members. When proposing new work items from International 2020, July 2020
Japan or developing and proposing working drafts, we take [3] Azuma, M., et al: Software Quality Evaluation Guidebook, Japan
in the results of practical application and opinions from Standards Association, 1994
industry and users. In WG 6, future watch type activities as [4] Ministry of Economy, Trade and Industry: Software Metrics Advanced
the following have been performed for seeking new market Project, http://www.meti.go.jp/policy/it_policy/softseibi/
and technology. [5] Tsuda, N., et al: WSQF: comprehensive software quality evaluation
framework and benchmark based on SQuaRE, Proceedings of the 41st
a) Setting up WG 6-led Ad-hoc group International Conference on Software Engineering, pp 312–321, 2019
[6] WSQB17: Waseda Software Quality Benchmark,
b) Utilizing SC7 level topics future watch(SC 7/AG 5) http://www.washi.cs.waseda.ac.jp/wsqb/