=Paper=
{{Paper
|id=Vol-3612/IWESQ_2023_Paper_01
|storemode=property
|title=Application of ISO/IEC25000 (SQuaRE) Series to SI Projects
|pdfUrl=https://ceur-ws.org/Vol-3612/IWESQ_2023_Paper_01.pdf
|volume=Vol-3612
|authors=Hiroyuki Kawai,Akiyasu Yamada
|dblpUrl=https://dblp.org/rec/conf/apsec/KawaiY23
}}
==Application of ISO/IEC25000 (SQuaRE) Series to SI Projects==
https://ceur-ws.org/Vol-3612/IWESQ_2023_Paper_01.pdf
Application of ISO/IEC25000 (SQuaRE) Series to SI
Projects
Hiroyuki Kawai1, Akiyasu Yamada1
1 NEC Corporation, 5-7-1 Shiba, Minato-ku, Tokyo, 108-8001, Japan
Abstract
Our department has the role of promoting reform in the SI business of the NEC Group and improving
the quality and productivity of products and services. To this end, we are working to manage technology
and distribute know-how across organizations in relation to software and system production. This
includes the drafting and implementing of policies for software and system engineering such as
development methodologies.
As quality requirements for IT systems become increasingly complex and diversified, the need is felt for
a mechanism that can logically and objectively explain quality. One means of perceiving quality is to
apply the ISO/IEC 25000 (SQuaRE) series of international standards, but at present, they are not being
sufficiently used on-site in IT system construction projects. Going forward, it will be necessary to
perceive quality not only from the viewpoint of process quality but also in terms of product quality based
on customer concerns. In this paper, we report on the results of our activities in testing the effectiveness
of applying the ISO/IEC 25000 (SQuaRE) series to IT system construction projects and in devising ways
of applying them.
Keywords
Software quality, quality evaluation, software development, System Integration, SQuaRE 1
1. Quality Management Trends construction of IT systems, it can be seen that factors
like sudden changes in IT technologies and short
in IT System Construction delivery times are making it difficult to properly apply
quality management techniques that use “statistical
In recent years, IT systems have become such an bug prediction and management techniques” specific
indispensable part of people’s lives that failures in to type of development project, type of language, and
those systems have turned into major social problems. type of organization taking the number of bugs to be a
This state of affairs has forced the providers of IT prime indicator. Techniques that place importance on
systems and services to be held accountable and process quality aim to achieve an indicator value in
explain why those failures occurred. terms of the number of bugs, but this results in a
At the same time, the role of IT systems in the situation in which quality is managed only from the
corporate world is shifting from being just a tool to viewpoint of software developers.
being a form of management itself, and as a result, the At present, quality awareness is rising among
requirements placed on IT systems are becoming customers, so product quality that presumes process
increasingly complex and diversified. Customers are quality management is being reconsidered and the
becoming increasingly aware of quality, and a trend is need is being felt for quality management techniques
emerging in which customers themselves are that focus on quality of interest to customers.
performing objective evaluations of IT system quality
such as by using outside process management
companies for development processes and quality 2. Definitions of Quality of
management. Interest to Customers
In Japan, the Information-technology Promotion
Agency (IPA) has made recommendations on the need Quality of interest to customers is not limited to the
for suppliers of software products to explain software presence or absence of bugs but covers a wide range of
quality to users premised on the ISO/IEC 25000 issues such as effectiveness in business tasks, ease of
(SQuaRE) series of international standards (referred system use, and reliability. Definitions of these various
to below as the “SQuaRE series”) [1]. aspects of quality have been made such as the Kano
Additionally, on examining the situation at System model [2], but as definitions of quality of interest to
Integration (SI) project sites involved in the
Proceedings IWESQ 2023,5th International Workshop on
Experience with SQuaRE series and its Future Direction, December 4,
2023, Seoul, Korea
EMAIL:h.kawai@nec.com (Hiroyuki Kawai);
akiyasu-yamada@nec.com (Akiyasu Yamada)
© 2023 Copyright for this paper by its authors. The use permitted under
Creative Commons License Attribution 4.0 International (CC BY 4.0).
CEUR Workshop Proceedings (CEUR-WS.org)
CEUR
ceur-ws.org
Workshop ISSN 1613-0073
Proceedings
3
customers, we here focus on using the SQuaRE series
to achieve explanations of quality for third parties. 1. The objective is not strict classification or
The SQuaRE series, however, are international exhaustive use of quality characteristics
standards for quality requirements of software /subcharacteristics
products and for evaluating those requirements To begin with, the quality models of the
targeting software overall, and as such, suffer from the SQuaRE series cover a very wide range of
following problems. quality characteristics and the need may be
felt for using all quality characteristics
• The SQuaRE series cover a wide range of /subcharacteristics in an exhaustive manner.
standards and target a variety of software products, However, it is often the case in SI projects that
so the descriptions in those standards tend to be resources that can be allocated (cost) and
generic in nature. As a result, project personnel delivery date are highly prioritized in addition
engaged in IT system construction encounter to target quality. Consequently, it is realistic to
expressions that they are not familiar with and find just use these quality characteristics
difficult to understand. /subcharacteristics as a viewpoint in setting
• Among Quality, Cost, and Delivery (QCD) in SI priorities for quality of interest to customers
projects, cost and delivery are highly prioritized, while keeping a QCD balance in mind (they
and there are many cases in which they are decided can guide the thinking of personnel and be
on in advance, which makes it difficult to secure the used as a reference for prioritizing quality
cost and time to study and deal with the quality requirements).
characteristics/subcharacteristics of all the quality Next, the objective of many SI projects is not
models in the SQuaRE series. to obtain ISO third-party certification, so strict
• Applying the general-purpose SQuaRE series classification of quality characteristics
of standards to SI projects requires tailoring them /subcharacteristics is not recommended
to the actual circumstances surrounding a (increases costs).
particular project along with a thorough
knowledge of SQuaRE (we ourselves spent about 2. Systematic use of the SQuaRE series from the
one year in achieving an understanding of quality upstream in consensus building with
models in the SQuaRE series and holding customers
discussions on their application to SI projects). The SQuaRE series can be used to form an
agreement on vendor quality requirements
with customers and stakeholders in the
project proposal and planning stage.
3. Learning about Quality Quality management based on the
Models and their Application management of indicator values such as
number of bugs and review time are process-
to SI quality centric with respect to work results
from reviews, tests, etc. In contrast, by
On applying the SQuaRE series to SI project sites, we focusing on product quality of interest to
were aware of the above problems, but we began with customers, we believe that applying the
“definitions of quality of interest to customers” in IT SQuaRE series to a project in a balanced
system construction using the Quality Model Division manner from the viewpoint of product quality
(ISO/IEC 2501x) of the SQuaRE series. and managing quality from multiple
To begin with, we set up a study team to learn perspectives in the upstream—where many
about the basics of the SQuaRE series using Japan’s JIS waterfall projects incorporate quality—can
standards [3], IPA’s “Software Quality Guide for a provide an IT system of even higher quality.
Connected World” [4], and the results of academic
research [5][6][7]. In this way, we made progress in 3. Use in analysis when quality problems occur
achieving a mutual understanding of SQuaRE, but at (not recommended)
the same time, there was some variation among team We believe that the quality model framework
members in how to interpret the SQuaRE series, and of the SQuaRE series can also be used for
this sometimes impeded discussions. projects in progress or existing deliverables
Additionally, for the quality characteristics from the viewpoint of checking for excesses or
/subcharacteristics of the ISO/IEC 25010 (product deficiencies from a quality perspective along
quality) and ISO/IEC 25012 (data quality) quality the way. From the beginning, however, it has
models, we discussed our understanding of each of been important to execute a project by turning
those characteristics/subcharacteristics (57 quality quality requirements into specifications in
characteristics in all) and examples of interpreting consultation with customers and stakeholders
them for application to SI projects. from the project proposal/planning stage and
These discussions were held over a period of about reaching a consensus on assigning priorities
one year and included an exchange of opinions with a and making measurements and evaluations.
certain outside vendor that had experience in dealing In this context, we do not recommend using
with the SQuaRE series. Through these activities, we the SQuaRE series when quality problems
discovered the following policies in using the SQuaRE occur.
series of standards.
4
4. Using the relationships between quality
characteristics ⚫ Provide an SI-oriented explanation based on
Relationships exist between ISO quality SQuaRE definitions of quality characteristics
models and quality characteristics /subcharacteristics (original text)
/subcharacteristics as reported in IPA’s ⚫ Illustrate quality characteristics
“Software Quality Guide for a Connected /subcharacteristics in line with situations in
World” [4] and in the research of academic IT system construction in a way that SI
institutions [8]. project personnel can easily visualize those
We present the following guidelines to characteristics/subcharacteristics
effectively use these relationships between ⚫ Illustrate relationships between quality
quality characteristics in SI projects characteristics (positive effects, negative
(described in detail later). effects, quality derivation) and explain how
to use them in an SI project
⚫ Positive effect: Improves the return on
investment of an SI project Specifically, we created a guide that, assuming SI
⚫ Negative effect: Results from projects, includes definitions of quality characteristics
implementing risk countermeasures in /subcharacteristics in the quality models of the
the operation of an SI project SQuaRE series and explanations of those definitions
⚫ Quality derivation: Used in deriving from IPA’s “Software Quality Guide for a Connected
related quality requirements (makes World” [4] plus interpretations of those quality
work more efficient and improves the characteristics/subcharacteristics assuming SI
accuracy of quality targets) projects at NEC (Figure 1: Example of a quality model
guide for SI project personnel).
Along with the above, we have added illustrations
4. Explanation for SI Project of quality subcharacteristics in NEC SI projects to help
Practitioners personnel in SI projects understand product quality
models (Figure 2: Example of a quality model guide for
SI project personnel (illustration of quality
We investigated how to efficiently convey the results
characteristics)).
of our study team activities to personnel at SI project
sites and prepared a guide based on the following
policies.
Figure 1: Example of a quality model guide for SI project personnel
5
Figure 2: Example of a quality model guide for SI project personnel (with illustration of quality characteristics)
This guide also presents samples of the Next, there are positive effects and negative
relationships between quality characteristics effects between quality characteristics
/subcharacteristics of different ISO quality models as /subcharacteristics. Given certain quality
reported by IPA’s “Software Quality Guide for a characteristics/subcharacteristics having a
Connected World” [4] and the research of academic relationship, a positive effect occurs when improving
institutions [8] to help personnel understand those the quality of one characteristic/subcharacteristic has
relationships. a good effect on the quality of the other. In contrast, a
First, recognizing that the following types of negative effect occurs when improving the quality of
relationships exist between quality characteristics of one characteristic/subcharacteristic has a bad effect
different quality models, we present an example on the quality of the other.
(Figure 3: Example of relationships between quality For example, improving response time and
characteristics of different quality models). raising performance so that screen transitions become
smoother and user operability improves is considered
1. Achieved with product quality model to be a positive effect. On the other hand, introducing
Quality characteristics that achieve quality during two-factor authentication to improve security
use can be used to derive quality characteristics in increases the number of screens to be checked and the
the product quality model. These relationships operations needed to reach the function one wants to
can be used as reference when incorporating use. This reduces user satisfaction and is therefore a
quality from the user’s point of view in specific negative effect.
quality characteristics in the product quality We created a matrix showing these relationships
model. and presented those relationships together with
grounds explaining them (Figure 4: Relationship
2. Supports quality during use matrix between quality characteristics (Compatibility),
In the case that quality in the product quality Figure 5: Grounds explaining relationships between
model is studied first, these relationships can be quality characteristics (Interoperability)).
used as reference to study target quality during We therefore believe that focusing on these
use in a retroactive manner. relationships between quality characteristics and
making designers aware of them can produce
3. Clarification of division between functions and specifications for quality requirements that achieve a
data balance in a more efficient way.
The division between quality achieved with However, the interpretation of quality
functions (that include data) and quality achieved characteristics and the relationships among them may
by data only can be clarified. These relationships change according to the nature of the target project, so
can be used as reference when, for functions, an tailoring them to the project is possible.
awareness of data characteristics is needed, and
for data, when identifying characteristics to be
linked with a function.
6
Figure 3: Example of relationships between quality characteristics of different quality models
Figure 4: Relationship matrix between quality characteristics (Compatibility)
7
Figure 5: Grounds explaining relationships between quality characteristics (Interoperability)
5. Evaluation of the Use of ⚫ Software as a Service (SaaS) development
project
Quality Models in SI Projects PoC execution phase: planning phase
⚫ IT system construction project including
We ourselves conducted proof of concept (PoC) trials construction of machine facilities, etc.
for several projects to gain insights from a quality PoC execution phase: basic design phase
perspective in executing the project and to check the ⚫ Government-related IT system construction
effectiveness of risk management. In these trials, we project
used the viewpoints expressed by quality models in PoC execution phase: basic design phase
the SQuaRE series of standards in actual SI projects
and classified and organized quality requirements
(explicit/implicit needs).
5.3. PoC Procedure
On performing a PoC trial, we ourselves as persons
5.1. Purpose of PoC knowledgeable about the SQuaRE series classified
and analyzed quality requirements based on
Our purpose in conducting these PoC trials was to materials provided from each project.
test the following hypotheses.
1. Visualization of quality status by quality
1. Visualization of quality requirements using classification
quality models Using project documents including the proposal
Can quality requirements that must be and definitions of requirements, we classified
satisfied by a project be further clarified by quality based on quality models of the SQuaRE
classifying quality in terms of quality series and visualized the quality status of the
characteristics/subcharacteristics? target project.
⚫ Can the state of quality be clearly 2. Analysis of quality status
recognized compared with that before We clarified the quality problems (including
quality classification? speculations) and countermeasures that should
⚫ Can excesses or deficiencies in quality be considered in the target project from quality
requirements be noticed? status classified as described above and from
relationships between quality characteristics
2. Checking effectiveness in an SI project (based on ISO definitions, research results, and
Could measures be taken to avoid project risk SI considerations).
from quality problems extracted from the
relationships among quality characteristics? 3. Project-directed proposals
Based on the results of analysis, we proposed
the following countermeasures (including
5.2. Target Projects speculations) such as risk hedges in executing a
project.
The projects targeted for PoC trials are
summarized below. The time period of each trial ⚫ Proposed that the presence of excesses or
was about a month and a half to two months deficiencies in terms of completeness be
according to each project’s schedule.
8
checked from a quality perspective (prevent requirements and analyzed why those trends
omission of requirements). occurred (Figure 6: Sample of project quality
⚫ “Deficiencies” from a quality perspective report (example of visualizing trends in quality
may indicate implicit needs, so we proposed characteristics) and Figure 7: Sample of project
that deficient points be checked to see quality report (example of analyzing trends in
whether they are indeed implicit needs. quality characteristics)).
⚫ Proposed that “outside the scope” from a Next, among various quality characteristics
quality perspective may be excluded from (explicit/implicit), we visualized quality
the project (eliminate waste). characteristics that have little mention in
⚫ Proposed that quality goals be explained to system requirements and proposed measures
stakeholders including customers using a for future projects from the relationships
visualized quality perspective (objective between quality characteristics, etc. (Figure 8:
explanation of quality). Sample of project quality report (analysis of
trends in quality characteristics: quality
characteristics of concern) and Figure 9: Sample
5.4. Main Deliverables of project quality report (example of quality
characteristics that a project should consider)).
1. Quality status report (Figure 6: Sample of
project quality report) 2. Quality perspective checklists by type of design
This deliverable reported on the classification specification
and analysis of quality requirements, proposals This deliverable for the basic design phase
for risk countermeasures to be taken by the includes design perspectives and review points
project from the perspective of quality that take into account quality characteristics.
characteristics, etc. Specifically, we focused on
groups of related quality characteristics based
on project characteristics from system
Figure 6: Sample of project quality report (example of visualizing trends in quality characteristics)
9
Figure 7: Sample of project quality report (example of analyzing trends in quality characteristics)
Figure 8: Sample of project quality report (analysis of trends in quality characteristics: quality
characteristics of concern)
10
Figure 9: Sample of project quality report (example of quality characteristics that a project should consider)
* “Measures in this project” relate to individual project and are not displayed here.
doubts as to whether such an amount of checklists
could be realistically checked.
5.5. Results from the PoC Consequently, to enable the use of checklists
Verification from a quality perspective, we determined that it
would be best if SI project personnel were to first
The following results were obtained for each of the test obtain an understanding of the quality models
objectives in the PoC verification. and quality characteristics/subcharacteristics of
the SQuaRE series and to then use those checklists
1. Visualization of quality requirements using at the beginning of each phase with the aim of
quality models checking off specific design points.
Classifying and visualizing system requirements
based on ISO quality models made it possible to
grasp trends in the current state of the project and 6. Conclusion
locations that require countermeasures while
increasing the resolution for studying quality Through activities that lasted for a year and a half, we
improvements. found the following effects to be true by applying the
SQuaRE series of standards to SI projects involved in
2. Checking effectiveness in an SI project the construction of IT systems.
We were able to offer new insights for a project by
mentioning things initially not noticed such as the ⚫ The visualization of quality requirements
possibility of degrading a subsequent process or (classifying and presenting quality status) is
negatively affecting other characteristics. effective in further clarifying quality status.
⚫ The application of SQuaRE series is effective in
3. Other improving return on investment and avoiding
By proposing not only general measures for risk in achieving project QCD.
quality characteristics but also measures that
consider project characteristics, we could In the above way, we confirmed that explaining IT
increase motivation for risk avoidance in the system quality in SI projects is effective, but as
project. described in policies on using the SQuaRE series in SI
projects, building a consensus with the customer from
4. Things that could not be confirmed the project proposal and planning stage is important
Since an understanding of the quality models and and that the SQuaRE series should be used as a manual
quality characteristics /subcharacteristics of the or bible for project quality during project execution
SQuaRE series would normally be assumed to and up to project completion.
complete quality perspective checklists for This can be taken to mean “quality strategy,” and
different deliverables, only check items with going forward, we plan to use not only the Quality
abstract expressions were created for personnel Model Division (ISO/IEC 2501x) of the SQuaRE series
in PoC projects, so effects could not be confirmed. but also the Quality Measurement Division (ISO/IEC
However, if a checklist is created that assumes no 2502x), Quality Requirements Division (ISO/IEC
knowledge at all of the SQuaRE series, a huge 2503x), Quality Management Division (ISO/IEC
amount of checklists including a variety of 2500x), and Quality Evaluation Division (ISO/IEC
explanations would be created and there would be 2504x). Furthermore, in addition to SI projects using a
11
waterfall type of development, we plan to apply the
SQuaRE series to DevOps (SI projects of the type in
which factors such as agile development, sudden
changes in IT technologies, and short delivery times
make it difficult to properly apply “statistical bug
prediction and management techniques” specific to
type of development project, type of language, and
type of organization taking the number of bugs to be a
major indicator).
7. References
[1] Information Technology Promotion Agency
(IPA), Japan, “Guidelines for implementing
software quality explanation scheme,”
(https://www.ipa.go.jp/archive/digital/iot-en-
ci/mieruka/software.html) (In Japanese)
[2] Noriaki Kano, Nobuhiko Seraku, Fumio
Takahashi, Shin-Ichi Tsuji, Japan, “Attractive
Quality and Must-Be Quality,” Quality, 14, No. 2,
pp. 39-48, 1984. (In Japanese)
[3] JIS X 25000: 2010 Systems and software Quality
Requirements and Evaluation (SQuaRE) – Guide
to SQuaRE
[4] Information-technology Promotion Agency (IPA),
Japan, “Software Quality Guide for a Connected
World,”
(https://www.ipa.go.jp/publish/tn20150529.ht
ml) (In Japanese) (In Japanese)
[5] Motoei Azuma, Japan, “History and Overview of
Systems and software Quality Requirements and
Evaluation (SQuaRE) Series of Standards,” SEC
Journal, Vol. 10, No. 5 (2015). (In Japanese)
[6] Tsuyoshi Nakajima, Japan, “Applying Quality
Requirements Framework to an IoT System and
its Evaluation,” SIGSE202 (2019). (In Japanese)
[7] Ministry of Economy, Trade and Industry (METI),
Japan, Investigative Report on Measure for
System/Software Product Quality Requirement
Definition and Evaluation.
[8] Waseda University, Japan, RISE (Research
Initiative on Advanced Software), “Framework
for Software Quality Quantification and
Comprehensive Quality Evaluation,” 2015. (In
Japanese)
12