- ISO/IEC JTC1/SC7 WG6 launched a study group on the future direction of the SQuaRE series, whose purpose is to discuss and establish a future plan of the SQuaRE series. This paper describes the proposed future plan, including provision of guidelines to use the series, adaptation of it to new technologies, and governing the terms & definitions and evolution of quality measures to keep the series simple and consistent.
• correcting flaws, gaps and integrity issues of the current documents, and • catching up the advent of new technologies.
This paper presents the outcome of this study group. Section II shows the overview of the reports, Section III and Section IV describe proposals for technical and governance issues. Section V concludes this paper.
The current information systems are drastically changed: More integration based (development -> combination of services consumed and components) • Increase of importance of data
Unclear responsibility for its quality (laid in several organization)
These changes make quality more and more critical for developing current ICT products and services. The SQuaRE series should provide a useful framework to achieve its quality goals. Based on the analysis of SQuaRE user feedback and discussion among the study group members, the following recommendations are made to do so:
Adapting the SQuaRE series to new technologies. •
Guiding quality requirements analysis, quality engineering and quality evaluation for: Ø Ø
Various types of systems and IT services
Applying to various development processes • Establishing a mechanism to evolve and maintain SQuaRE series for terms and definitions, and evolution of quality models and measures.
III. PROPOSALS FOR TECHNICAL ISSUES
(
[Problem]
The SQuaRE series does not have a division related to engineering (realization of requirements and stepwise verification in various development phases). Therefore, even though the developers understand the importance of quality engineering, they cannot effectively use the series.
How to implement quality requirements (assignment and verification) Designing quality processes (inclusion of activities for quality implementation and verification in the development process)
functions, and design policies elements, architecture, Knowledge management and traceability of quality requirements and baselines
Because the above items are beyond Management
Division (2500n), new division for quality engineering is
needed.
(
[Problem]
The current quality evaluation division consists of the definition of general-purpose evaluation process (ISO/IEC 25040), and the usage guideline (ISO/IEC 25041) for each user (developer, acquirer, and independent evaluator) .There is a great demand from the industry for methods and techniques to support how to plan inspections and testing on quality, and concrete assessment based on their results. In addition, due to the development of ISO/IEC 2502n and the revision of ISO/IEC 25030, the quality requirements and their measurement have been clarified, so modifications aligning with these will also be necessary. [Note]
WG6 already decided to enhance the quality models themselves, first starting from quality in use model (ISO/IEC 25010-part 3) and product quality (ISO/IEC 25010-part 2). The study group provides needs for future revision of the quality models including them, in Annex C. • It is necessary to carefully examine each case. (whether it is an application of the existing model or needs new one) Quality consideration for products or IT services using new technologies should be reported in TRs and to provide guidance to apply the models to them, keeping the quality models stable (Fig. 1). • Ensuring consistency with 2502n and 25030R • Improving the concept of evaluation modules (EVs) (and encouraging industries to provide ANNEXs) •
Guidelines for the following activities: Ø Organizing quality testing including inspections, aligning with ISO/IEC 29119 (WG 26) Ø Comprehensive quality evaluation (e.g., for judgment of delivery) based on measurement results ² How to devise a set of quality measure
suitable for evaluation ² Concept of evaluation(analysis of testing
results, etc.) and rating Ø Selecting the right quality characteristics from some evaluation goal Ø Choosing an appropriate evaluation module for the characteristics or to make a new evaluation module [Note]
Definition of the format for evaluation modules, and
examples of them in ANNEXs are useful, although it should
have some improvements, for instance, ISO/IEC 25020
ANNEX C should be referenced from it.
(
[Problem]
Most of the measures in ISO/IEC 25023 are quality-in-use measures since the specified measures are about external properties at runtime. There are several coding standards such as MISRA, AUTOSAR, and CISQ, which provide the checklists or rules for codes to entail quality measures. For SQuaRE to be considered a strong guide for measurement of software and systems product quality, it must improve how it guides for quality measurement of internal properties.
Guideline for quality measurement, especially for
internal properties of codes
Guideline for application of ISO/IEC 2502n (Data
measurement amount: UNINFO guideline)
=>ISO/IEC 25052 (b)
(
[Problem]
When applying the (existing) quality models in a highnecessity but relatively narrow area, the degree of detail and the coverage range of them does not fit perfectly and so it may be difficult to use as it is. It is necessary to standardize how to use the quality models for those areas, or if necessary, how to define tailored quality models. Ø Application interface: relating to quality in use and product quality Ø AI component: data quality and product quality Ø Hardware component • • • • Ø More important characteristics and measures for the technology Ø Examples of application of SQuaRE models Before ISs and TRs, it might be better to publish them using Web column and so on. • The consistency of “document quality” with document-related standards handled by the WG 2 (in which quality of user documents and online helps is not addressed) should be considered.
Better to go to the extension division of 2507n.
(
Application to various systems and XaaS [Problem]
There are a lot of types of systems, which is desirable to present guidelines and examples on how to cover the spread of the type of target systems and services. [Proposed solution]
How to deal with the diversity of systems and services
Ø Interpretation of the quality requirements
framework (ISO/IEC 25030) and the quality
evaluation processes when they are applied to IoT
systems and system of systems (Method of
gradual refinement of target)
[Note]
[Problem]
Application to XaaS may be a deployment of ISO/IEC 25051.
(
Although ISO/IEC 25030 and 25040 provide generic and abstract processes and procedures, there are difficulties in how to apply them to actual development. It is necessary to guide how to apply them to various process models (from the viewpoint of procedures, notes).
Waterfall, iterative, evolutionary: Organization viewed from process • System and Software product line (application to variability in general)
[Note] • Process are defined in ISO/IEC
ISO/IEC/IEEE 15288. 12207 and Guidelines at the TR level, starting from the high demand areas first, would be better.
Concerning agile or component-based development, guidelines for showing at what timing to define quality requirements and evaluate.
We should develop the idea in WG 6 and then discuss
it with WG 7 and WG 24.
(
Application to certification of products / components [Problem]
When products or components are used in a larger system as black boxes, it is desirable that the quality requirements for them are defined and guaranteed on them. This may lead to product quality certification. Quality certification may be a big area to use SQuaRE, which has already been implemented in several countries. It is useful to provide a framework for a better quality certification system. • IS on authentication scheme can hardly be realized, because ISO / CASCO are extremely nervous about the word “authentication”. It would be better to provide a form of guideline. • ISO/IEC 25051 merely stipulates requirements for test documents and instructions for conformity assessment. • • • • • •
IV. PROPOSALS FOR GOVERNANCE ISSUES
The number of the SQuaRE documents has increased to 18, and the number of their terms and definitions is now 487. A document has many copies of its terms and definitions from the other documents of SQuaRE or the other standards, which causes some inconsistencies (the same concept for different words, vice versa).
Make ISO/IEC 25000 a free IS. Maintain this on a regular basis.
Continually maintain the central repository of the terms, and make it publicly accessible. • Eliminate duplication of the terms for standard development / revision. To do so, make a guideline to do so. [Note] • Process are defined in ISO/IEC
ISO/IEC/IEEE 15288. 12207 and Guidelines at the TR level, starting from the high demand areas first, would be better.
Concerning agile or component-based development, guidelines for showing at what timing to define quality requirements and evaluate.
We should develop the idea in WG 6 and then discuss
it with WG 7and WG 24.
(
[Problem]
There are cases where the ISO/IEC 2502n measures cannot be used without some modification or adaptation because of their mismatch with the user needs for the measurement. These cases include ones using new technologies (e.g. Big Data, AI, IoT, and Blockchain) and ones using existing technologies with new needs for measurement.
Here follows a possible approach that in principle can fulfill every needs of measurements that exploits the possibilities of conforming measure mechanism defined in Annex B of ISO/IEC 2502n.
Measuring needs covered by 2502x standard
measures?
Measuring needs covered by registered 2502x compliance measures?
No
No Define and register new 2502x compliance measures Use the measures
End
Yes Yes
The proposed future changes described above, including quality engineering division, need to reorganize the structure of SQuaRE, which shall be understandable for the SQuaRE readers and refrain ad hoc creation of ISs, TRs and TSs. Fig. 3 shows the current structure of the SQuaRE series. [A1] Layered
Based on the quality model department, quality measurement divisions can be defined, and each division (requirement, engineering, evaluation) of quality activities can be made using them. Above them comes the quality control division that controls the quality activities. Merits: Incorporating the engineering department. The usage relationship is clear.
Demerits: Currently new engineering division is not scheduled. Such large change make be confusing to the readers. [A2] Conservative version
The quality model division comes to the center of the square, and the others use it. Merits: The quality model centered architecture is comprehensible.
Demerits: Not prepared for where the engineering division will come. 2) Subdivisions of the extension division
Fig. 6 shows the proposed subdivisions of the extension division, which guides creation of new ISs, TRs and TSs, where 2509n is temporally excluded from the extension division.
This paper describes future plans for the SQuaRE series which the study group on SQuaRE future direction proposes, including provision of guidelines to use the series, adaptation of it to new technologies, and governing the terms & definitions and evolution of quality measures to keep the series simple and consistent.
The plans have been basically accepted and some of them already start to be carries out in WG6. However, we recognize the importance to continuously hear the needs for using SQuaRE from both researchers and practitioners on system and software quality.
The author wishes to thank Dr. Zhang Yangyang, Mr. Vijay Shankar Krishnamoorthy, Ms. Hyun-Chong Kim, Mr. Jaehyo Lee , Mr. Michael Gayle, Mr. Jonathan Roy, Dr. Bill Curtis, Dr. Domenico Natale, Mr. Andrea Trenta, Mr. Jean Louis Miche, Dr. Cai Lizhi, and Mr. Raúl Martinez for great contributions as members of the study group. APPEDIX: ISSUES ON APPLICATION OF SQUARE QUALITY
MODELS TO NEW TECHNOLOGIES Fig. A-1 shows a word matching-level correspondence between SQuaRE quality models and quality aspects of new technologies, which have been extracted from literatures relating to the technologies. Some potential problems exist, which need future investigation: • Some characteristics are not connected with any aspects of a new technology (The technologies do not really need them?) • Gaps of meanings between the quality aspects and
SQuaRE characteristics (Some inconsistency) • No counterparts in the SQuaRE models (SQuaRE has not dealt with them yet.)