=Paper=
{{Paper
|id=Vol-2545/invited1
|storemode=property
|title=Study Group Report on SQuaRE Future Direction (invited paper)
|pdfUrl=https://ceur-ws.org/Vol-2545/invited-01.pdf
|volume=Vol-2545
|authors=Tsuyoshi Nakajima
|dblpUrl=https://dblp.org/rec/conf/apsec/Nakajima19
}}
==Study Group Report on SQuaRE Future Direction (invited paper)==
https://ceur-ws.org/Vol-2545/invited-01.pdf
Study Group Report on SQuaRE Future Direction
Tsuyoshi Nakajima
Dept. of Computer Science and Engineering
Shibaura Institute of Technology
Tokyo, Japan
tsnaka@shibaura-it.ac.jp
Abstract— ISO/IEC JTC1/SC7 WG6 launched a study • Guiding quality requirements analysis, quality
group on the future direction of the SQuaRE series, whose engineering and quality evaluation for:
purpose is to discuss and establish a future plan of the SQuaRE
series. This paper describes the proposed future plan, including Ø Various types of systems and IT services
provision of guidelines to use the series, adaptation of it to new
Ø Applying to various development processes
technologies, and governing the terms & definitions and
evolution of quality measures to keep the series simple and • Establishing a mechanism to evolve and maintain
consistent. SQuaRE series for terms and definitions, and evolution
of quality models and measures.
Keywords—SQuaRE, future plan, proposal, user feedback
III. PROPOSALS FOR TECHNICAL ISSUES
I. INTRODUCTION
The ISO/IEC 25000 (Systems and Software Quality A. Enhancement of core divisions
Requirements and Evaluation: SQuaRE) series are a set of (1) Quality engineering division (new)
international standards for systems and software, data and IT [Problem]
service quality. The first work for ISO/IEC 25000 (SQuaRE)
series has been almost done, which have been widely used in The SQuaRE series does not have a division related to
the industry for a couple of decades. In the research field, they engineering (realization of requirements and stepwise
have been also examined as a tool for performing various verification in various development phases). Therefore, even
analyses and evaluations on systems and software quality. though the developers understand the importance of quality
engineering, they cannot effectively use the series.
In order for its readers to use it more effectively and
efficiently, ISO/IEC JTC1/SC7 WG6 launched a one-year [Proposed solution]
study group on the future direction of the SQuaRE series in
2018, whose purpose is to discuss and establish a future plan • How to implement quality requirements (assignment
of the SQuaRE series. What was planned to do includes: and verification)
• getting feedback from the field for the current ISs/TSs • Designing quality processes (inclusion of activities for
quality implementation and verification in the
• correcting flaws, gaps and integrity issues of the development process)
current documents, and
• Quality deployment to elements, architecture,
• catching up the advent of new technologies. functions, and design policies
This paper presents the outcome of this study group. • Knowledge management and traceability of quality
Section II shows the overview of the reports, Section III and requirements and baselines
Section IV describe proposals for technical and governance
issues. Section V concludes this paper. • Quality validation and verification
[Note]
II. OVERVIEW OF THE REPORT
The current information systems are drastically changed: Because the above items are beyond Management
Division (2500n), new division for quality engineering is
• More integration based (development -> combination needed.
of services consumed and components)
(2) Quality evaluation division
• Increase of importance of data [Problem]
• Unclear responsibility for its quality (laid in several The current quality evaluation division consists of the
organization) definition of general-purpose evaluation process (ISO/IEC
These changes make quality more and more critical for 25040), and the usage guideline (ISO/IEC 25041) for each
developing current ICT products and services. The SQuaRE user (developer, acquirer, and independent evaluator) .There
series should provide a useful framework to achieve its quality is a great demand from the industry for methods and
goals. Based on the analysis of SQuaRE user feedback and techniques to support how to plan inspections and testing on
discussion among the study group members, the following quality, and concrete assessment based on their results. In
recommendations are made to do so: addition, due to the development of ISO/IEC 2502n and the
revision of ISO/IEC 25030, the quality requirements and their
• Adapting the SQuaRE series to new technologies. measurement have been clarified, so modifications aligning
with these will also be necessary.
Copyright © 2020 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
[Proposed solution] Ø AI component: data quality and product quality
• Ensuring consistency with 2502n and 25030R Ø Hardware component
• Improving the concept of evaluation modules (EVs) [Note]
(and encouraging industries to provide ANNEXs) • WG6 already decided to enhance the quality models
• Guidelines for the following activities: themselves, first starting from quality in use model
(ISO/IEC 25010-part 3) and product quality (ISO/IEC
Ø Organizing quality testing including inspections, 25010-part 2). The study group provides needs for
aligning with ISO/IEC 29119 (WG 26) future revision of the quality models including them, in
Ø Comprehensive quality evaluation (e.g., for Annex C.
judgment of delivery) based on measurement • It is necessary to carefully examine each case. (whether
results it is an application of the existing model or needs new
² How to devise a set of quality measure one)
suitable for evaluation • Quality consideration for products or IT services using
² Concept of evaluation(analysis of testing new technologies should be reported in TRs and to
results, etc.) and rating provide guidance to apply the models to them, keeping
the quality models stable (Fig. 1).
Ø Selecting the right quality characteristics from
some evaluation goal
Ø Choosing an appropriate evaluation module for
the characteristics or to make a new evaluation
module
[Note]
Definition of the format for evaluation modules, and
examples of them in ANNEXs are useful, although it should
have some improvements, for instance, ISO/IEC 25020
ANNEX C should be referenced from it.
(3) Quality measurement division
Fig. 1. Managing new technologies
[Problem]
• Each TR should include:
Most of the measures in ISO/IEC 25023 are quality-in-use
measures since the specified measures are about external Ø Brief explanation of the technology
properties at runtime. There are several coding standards such
as MISRA, AUTOSAR, and CISQ, which provide the Ø More important characteristics and measures for
checklists or rules for codes to entail quality measures. For the technology
SQuaRE to be considered a strong guide for measurement of Ø Examples of application of SQuaRE models
software and systems product quality, it must improve how it
guides for quality measurement of internal properties. • Before ISs and TRs, it might be better to publish them
using Web column and so on.
[Proposed solution]
• The consistency of “document quality” with
• Guideline for quality measurement, especially for document-related standards handled by the WG 2 (in
internal properties of codes which quality of user documents and online helps is
• Guideline for application of ISO/IEC 2502n (Data not addressed) should be considered.
measurement amount: UNINFO guideline) • Better to go to the extension division of 2507n.
=>ISO/IEC 25052 (b)
B. Application of SQuaRE
(4) Quality model division
(1) Application to various systems and XaaS
[Problem]
[Problem]
When applying the (existing) quality models in a high-
There are a lot of types of systems, which is desirable to
necessity but relatively narrow area, the degree of detail and
present guidelines and examples on how to cover the spread
the coverage range of them does not fit perfectly and so it may
of the type of target systems and services.
be difficult to use as it is. It is necessary to standardize how to
use the quality models for those areas, or if necessary, how to [Proposed solution]
define tailored quality models.
• How to deal with the diversity of systems and services
[Proposed solution]
Ø Interpretation of the quality requirements
• Guidelines to use quality models for: framework (ISO/IEC 25030) and the quality
evaluation processes when they are applied to IoT
Ø Application interface: relating to quality in use
systems and system of systems (Method of
and product quality
gradual refinement of target)
Ø Applicability to XaaS (SaaS，PaaS，IaaS) IV. PROPOSALS FOR GOVERNANCE ISSUES
Ø Block chain A. Enhancement of core divisions
[Note] (1) Terms and Definitions
Application to XaaS may be a deployment of ISO/IEC 25051. [Problem]
(2) Application to various processes The number of the SQuaRE documents has increased to
18, and the number of their terms and definitions is now 487.
[Problem] A document has many copies of its terms and definitions from
Although ISO/IEC 25030 and 25040 provide generic and the other documents of SQuaRE or the other standards, which
abstract processes and procedures, there are difficulties in how causes some inconsistencies (the same concept for different
to apply them to actual development. It is necessary to guide words, vice versa).
how to apply them to various process models (from the [Proposed solution]
viewpoint of procedures, notes).
• Make ISO/IEC 25000 a free IS. Maintain this on a
[Proposed solution] regular basis.
• Waterfall, iterative, evolutionary: Organization viewed • Continually maintain the central repository of the
from process terms, and make it publicly accessible.
• System and Software product line (application to • Eliminate duplication of the terms for standard
variability in general) development / revision. To do so, make a guideline
• Agile/DevOps to do so.
[Note] [Note]
• Process are defined in ISO/IEC 12207 and • Process are defined in ISO/IEC 12207 and
ISO/IEC/IEEE 15288. ISO/IEC/IEEE 15288.
• Guidelines at the TR level, starting from the high • Guidelines at the TR level, starting from the high
demand areas first, would be better. demand areas first, would be better.
• Concerning agile or component-based development, • Concerning agile or component-based development,
guidelines for showing at what timing to define quality guidelines for showing at what timing to define quality
requirements and evaluate.
requirements and evaluate.
• We should develop the idea in WG 6 and then discuss
• We should develop the idea in WG 6 and then discuss
it with WG 7 and WG 24. it with WG 7and WG 24.
(2) Evolution of quality measures
(3) Application to certification of products / components
[Problem]
[Problem]
When products or components are used in a larger system There are cases where the ISO/IEC 2502n measures
cannot be used without some modification or adaptation
as black boxes, it is desirable that the quality requirements for
because of their mismatch with the user needs for the
them are defined and guaranteed on them. This may lead to
product quality certification. Quality certification may be a big measurement. These cases include ones using new
area to use SQuaRE, which has already been implemented in technologies (e.g. Big Data, AI, IoT, and Blockchain) and
ones using existing technologies with new needs for
several countries. It is useful to provide a framework for a
measurement.
better quality certification system.
[Proposed solution] [Proposed solution]
• Guidelines for quality certification schemes • Here follows a possible approach that in principle can
=>ISO/IEC 25050 fulfill every needs of measurements that exploits the
possibilities of conforming measure mechanism
• Revision of ISO/IEC 25051 defined in Annex B of ISO/IEC 2502n.
[Note]
• IS on authentication scheme can hardly be realized,
because ISO / CASCO are extremely nervous about the
word “authentication”. It would be better to provide a
form of guideline.
• ISO/IEC 25051 merely stipulates requirements for test
documents and instructions for conformity assessment.
Start
Measuring needs Yes
covered by 2502x standard
measures?
No
Measuring needs
Yes
covered by registered 2502x
compliance measures?
No
Fig. 4. A1 (Layered) for the core divisions of the future SQuaRE
Define and register new 2502x
compliance measures
Merits: Incorporating the engineering department.
The usage relationship is clear.
Use the measures Demerits: Currently new engineering division is not
scheduled. Such large change make be confusing to
End
the readers.
Fig. 2. Flow of selecting quality measures / defining and registering new
ones [A2] Conservative version
[Note] The quality model division comes to the center of the
Some issues are to be investigated: square, and the others use it.
• How to register:
² interaction for registration between users,
NBs and ISO (WG6)
² accessing and conforming new measures
• Guidelines for determining conformance level of
measures
• Feedback to quality models if no existing
(sub)characteristics can support the new measure
Fig. 5. A2 (Conservative ones) for the core divisions of the future SQuaRE
(3) Reorganization of future SQuaRE series
[Problem] Merits: The quality model centered architecture is
The proposed future changes described above, including comprehensible.
quality engineering division, need to reorganize the structure Demerits: Not prepared for where the engineering
of SQuaRE, which shall be understandable for the SQuaRE division will come.
readers and refrain ad hoc creation of ISs, TRs and TSs. Fig. 3
shows the current structure of the SQuaRE series. 2) Subdivisions of the extension division
50 Fig. 6 shows the proposed subdivisions of the extension
Quality Model Division Various application
2501n division, which guides creation of new ISs, TRs and TSs,
60
where 2509n is temporally excluded from the extension
Extension Division
Usability
Quality Quality Quality
Requirements Management Evaluation 70-99
division.
Division Division Division
25030- 2500n 2504n
(No assignment)
Quality Measurement Division
2502n
Fig. 3. Current structure of the SQuaRE series
[Proposed solution]
1) Core division
Two acandidates (A1 and A2) for the core divisions
of the future SQuaRE series are proposed with their
merits and demerits. Fig. 6. Subdivisions of the extension division
[A1] Layered V. CONCLUSION
Based on the quality model department, quality This paper describes future plans for the SQuaRE series
measurement divisions can be defined, and each which the study group on SQuaRE future direction proposes,
division (requirement, engineering, evaluation) of including provision of guidelines to use the series, adaptation
quality activities can be made using them. Above them of it to new technologies, and governing the terms &
comes the quality control division that controls the definitions and evolution of quality measures to keep the
quality activities. series simple and consistent.
The plans have been basically accepted and some of them • Gaps of meanings between the quality aspects and
already start to be carries out in WG6. However, we recognize SQuaRE characteristics (Some inconsistency)
the importance to continuously hear the needs for using • No counterparts in the SQuaRE models (SQuaRE has
SQuaRE from both researchers and practitioners on system not dealt with them yet.)
and software quality.
ACKNOWLEDGMENT
The author wishes to thank Dr. Zhang Yangyang, Mr.
Vijay Shankar Krishnamoorthy, Ms. Hyun-Chong Kim, Mr.
Jaehyo Lee , Mr. Michael Gayle, Mr. Jonathan Roy, Dr. Bill
Curtis, Dr. Domenico Natale, Mr. Andrea Trenta, Mr. Jean
Louis Miche, Dr. Cai Lizhi, and Mr. Raúl Martinez for great
contributions as members of the study group.
REFERENCES
[1] ISO/IEC 12207:2008, Systems and software engineering - Software
life cycle processes
[2] ISO/IEC/IEEE 29148:2011 Systems and software engineering - Life
cycle processes - Requirements engineering
[3] ISO/IEC 25001:2014, Systems and Software engineering — Systems
and software product Quality Requirements and Evaluation (SQuaRE)
— Planning and management
[4] ISO/TS 25011:2017, Information technology — Systems and software
quality requirements and evaluation (SQuaRE) — Service quality
models
[5] ISO/IEC 25020:2007, Systems and Software engineering — Systems
and software product Quality Requirements and Evaluation (SQuaRE)
— Measurement reference model and guide
[6] ISO/IEC 25021:2012, Systems and Software engineering — Systems
and software product Quality Requirements and Evaluation (SQuaRE)
— Quality measure elements
[7] ISO/IEC 25063:2014, Systems and software engineering — Systems
and software product Quality Requirements and Evaluation (SQuaRE)
— Common Industry Format (CIF) for usability: Context of use
description
APPEDIX: ISSUES ON APPLICATION OF SQUARE QUALITY
MODELS TO NEW TECHNOLOGIES
Fig. A-1 Word matching-level correspondence between SQuaRE
Fig. A-1 shows a word matching-level correspondence quality models and quality aspects of new technologies
between SQuaRE quality models and quality aspects of new
technologies, which have been extracted from literatures
relating to the technologies. Some potential problems exist,
which need future investigation:
• Some characteristics are not connected with any aspects
of a new technology (The technologies do not really
need them?)