<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta>
      <journal-title-group>
        <journal-title>Future Direction, December</journal-title>
      </journal-title-group>
    </journal-meta>
    <article-meta>
      <title-group>
        <article-title>Digital Twin in Agile Development</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Tsuyoshi Nakajima</string-name>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Alessandro Simonetta</string-name>
          <email>alessandro.simonetta@gmail.com</email>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>Department of Enterprise Engineering University of Rome Tor Vergata</institution>
          ,
          <addr-line>Rome</addr-line>
          ,
          <country country="IT">Italy</country>
        </aff>
        <aff id="aff1">
          <label>1</label>
          <institution>Shibaura Institute of Technology</institution>
          ,
          <addr-line>3-7-5 Toyosu, Koto-ku, Tokyo, 135-848</addr-line>
          ,
          <country country="JP">Japan</country>
        </aff>
      </contrib-group>
      <pub-date>
        <year>2022</year>
      </pub-date>
      <volume>06</volume>
      <issue>2022</issue>
      <fpage>0000</fpage>
      <lpage>0002</lpage>
      <abstract>
        <p>Agile development is essential for efective digital transformation. However, since agile development tends to focus on functional implementation, the realization of true user value (quality in a broad sense) is often neglected. To support realization of true user value in agile, this paper proposes the concept of a quality digital twin, a quality representation of the target system synchronized at each iteration. The agile development focuses on the quality of target product itself more than that of processes and intermediate products, and therefore SQuaRE quality model can be much more applicable to the agile. It uses the SQuaRE quality framework for defining quality requirements, and engineering and evaluating the system. This concept enables the realization of a support system for user-driven, high-quality agile development. Its feasibility is demonstrated by showing system / data structure and usage of the quality digital twin.</p>
      </abstract>
      <kwd-group>
        <kwd>Agile</kwd>
        <kwd>digital twin</kwd>
        <kwd>quality management</kwd>
        <kwd>SQuaRE</kwd>
      </kwd-group>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>1. Introduction</title>
      <p>
        Digital Transformation is the transformation of
products, services, and business models based on the needs
of customers and society by responding to changes in
the business environment and utilizing information
technology [
        <xref ref-type="bibr" rid="ref1">1</xref>
        ]. Digital Transformation can perform high
value creation and development eficiency by repeating
the planning, prototyping, operation, and evaluation of
ideas quickly, with the user taking the initiative. Agile
development is one of the foundations required for this.
      </p>
      <p>However, because agile development tends to focus on
each sprint, it is possible to simultaneously progress the
quality estimation and evolve quality requirements.</p>
      <p>This paper presents the issues of quality achievement
in agile development and concept and design of the
quality digital twin.</p>
    </sec>
    <sec id="sec-2">
      <title>2. AGILE DEVELOPMENT AND ITS ISSUES</title>
      <p>2.1. Agile development
the implementation of functional requirements, the real- Agile software development refers to a group of software
ization and evaluation of quality requirements are often
neglected. To solve such a problem, we propose a concept
of a support system for user-driven high-quality agile
development (hereinafter called ”quality digital twin”) by
development methodologies based on iterative
development to deal with uncertainty and risk, as shown in 1 .</p>
      <p>Its important practical activities include short iterations
with a certain time window, continuous release
(automaapplying the digital twin concept to quality management tion, providing and evaluating working products), user
in agile development.</p>
      <p>
        The Quality Digital Twin has the quality state model as
its core, which models the quality of products in the agile
development. The model is based on the quality models
of the ISO/IEC 25010, including product quality models
and quality-in-use model [
        <xref ref-type="bibr" rid="ref2">2</xref>
        ]. By considering user value
with the quality-in-use model, refining and
implementing quality requirements/functions/architecture, and
iterating testing including non-functional testing, static
analysis, and quality evaluation based on user reviews in
nEvelop-O
4 th International Workshop on Experience with SQuaRE Series and its
and the most widely used one [
        <xref ref-type="bibr" rid="ref4">4</xref>
        ].
      </p>
      <sec id="sec-2-1">
        <title>2.2. Issues to user-driven agile development</title>
        <p>
          Digital transformation can perform high value creation
and development eficiency through rapid repetition of
user-driven idea planning, prototyping, operation, and
evaluation [
          <xref ref-type="bibr" rid="ref1">1</xref>
          ]. Therefore, agile development is one of the
necessity foundations for this. However, user
involvement is often limited to input of requirements and
evalaspects that the users are not directly benefited from is
often neglected to confirm. Agile development itself does
not have the support to overcome these problems and
achieve quality in a comprehensive and balanced manner.
        </p>
      </sec>
    </sec>
    <sec id="sec-3">
      <title>3. QUALITY DIGITAL TWIN</title>
    </sec>
    <sec id="sec-4">
      <title>CONCEPT</title>
      <sec id="sec-4-1">
        <title>3.1. Digital twin</title>
        <p>
          A digital twin (DT) is a living, intelligent and evolving
uation and review of deliverables at each development model, being the virtual counterpart of a physical entity
cycle, and users rarely are involved in decision-making or process for practical purpose, such as monitoring and
in managing the projects. As a result, agile development control, future prediction and planning, and conceptual
tends to derail from creating the value that users expect design and development [
          <xref ref-type="bibr" rid="ref7">7</xref>
          ].
for the cost [
          <xref ref-type="bibr" rid="ref5">5</xref>
          ]. From the results of the literature review [
          <xref ref-type="bibr" rid="ref7">7</xref>
          ] [
          <xref ref-type="bibr" rid="ref8">8</xref>
          ] [
          <xref ref-type="bibr" rid="ref9">9</xref>
          ], we
        </p>
        <p>We believe that one of the reasons why users cannot found that DT consists of all or a partial combination of
be involved in development decisions is that there is no the following five functions as shown in 2 (Generic form
quantitative and intuitive way to determine whether the of DT).
current quality of the target product is suficient for the
value they want to achieve, and what and how much is
missing.</p>
      </sec>
      <sec id="sec-4-2">
        <title>2.3. Quality management of agile development</title>
        <p>
          In the waterfall development, the quality of the target
product is defined, decomposing it into quality activities
and their goals for intermediate products in each develop- Figure 2: Generic form of Digital Twin
ment phase. The project tries to achieve them to assure
the quality of the target product indirectly. In contrast,
in the agile development, the quality to be achieved is
mainly the value demanded by the user, and its resources
are concentrated on creating an executable product to
realize and confirm the value. The quality is achieved
through repeated product evaluations with user
participation at each iteration [
          <xref ref-type="bibr" rid="ref6">6</xref>
          ]. From a quality management
perspective, agile development, compared to the
waterfall, has the advantage that its quality goals are
businessoriented and the way to check their achievement is direct.
        </p>
        <p>Modeling languages such as use cases in UML may
mitigate the above weakness of waterfall development
to some extent because they are easy to understand even
for non-experts. However, the quality of these models
does not directly imply the quality of the system, since
they only can define the functionality of the system, not
its quality.</p>
        <p>In the agile development, on the other hand, user
values are defined by the functionality the product should
have. Therefore, the development team tends to focus on
realizing the functionality, not user values themselves.</p>
        <p>To prevent this, the user review is conducted to evaluate
the product at the end of each iteration. However,
because the user review tends to be subjective, some quality
• Model synchronization: Synchronization of the
real-world target and the virtual world model
from data sensed from the real world one
• State estimation and prediction: Estimation and
prediction of target internal states from the model
(for monitoring)
• Controlling: Controlling the real-world target</p>
        <p>based on 2)
• Modelling and simulation: Future forecasting and
planning through simulation and optimization by
changing inputs and control parameters
• Information display: Display of information
about the monitoring of the estimated internal
state of the target, the results of future
predictions, or the provision of additional information
tied to the model.</p>
      </sec>
      <sec id="sec-4-3">
        <title>3.2. Application of DT concept to quality management in agile development and three hypotheses</title>
        <p>In waterfall development, quality is estimated using the
results of quality assurance activities (often processes)</p>
        <p>In agile development, requirements continuously
evolve. In other words, the quality digital twin is
evaluated in every iteration to accumulate necessary quality
requirements for its completion in the product backlog.</p>
        <p>Quality decomposition is performed using the
framework of Scrum: initiative, epic, and story, depending on
the scale of the project. 3 shows an example of quality
decomposition. First, user values are defined at the top
level as quality-in-use (QiU) requirements, and then
decomposed into product quality requirements, functions,
architectural design, or non-functional testing. Problems
in product quality discovered during development are
stored in the backlog as technical liabilities.</p>
        <p>Furthermore, we believe that the quality digital twin
would allow users to not only provide input and feedback
to the agile development but also take the initiative in
decision making on it. This is because the quality digital
twin has the potential to provide users for intuitive and
quantitative quality estimation in a way that only quality
assurance professionals have been able to do (Hypothesis
3).</p>
      </sec>
      <sec id="sec-4-4">
        <title>4.2. Quality requirements in agile development</title>
        <p>At the sprint planning, the decomposition of quality
requirements to be implemented into stories is performed
to put in the sprint backlog. During the execution of a
sprint, the results of testing and static analysis are
collected. Technical debt identified by the developer is also
collected into the product backlog.</p>
        <p>The sprint review is based on the collected
qualityrelated data to determine the current quality. In this case,
based on the decomposition at the time of planning, the
results of testing and static analysis are automatically degree to which the targeted quality have been achieved.
calculated in terms of their “contribution” to the quality Concerning product quality, we can see the balance of
characteristics/sub-characteristics. its achievement.</p>
        <p>
          During a sprint review, the quality dashboard displays
the quality evaluation results in a visual and
easy-tounderstand manner for the users to intuitively monitor 5. QUALITY STATE MODEL AND
the quality. The qualitydashboard shows the relationship ITS APPLICATION
between quality requirements, quality realization, and
quality activities, provides the ability to edit it ( 8 ), and This section presents ideas for the organization and use of
monitors the status of product quality. 4 shows an image the quality state model, which is a central part of Quality
of the quality status monitoring function. The achieve- DT.
ment percentage of all the QiU requirements is displayed
in the left-hand side of 4, in which QiU requirements are 5.1. SQuaRE quality models
listed in order of their importance (three sizes of length:
most important, important, and normal). In the right pie The ISO/IEC 25000 (SQuaRE) series provides a
framechart in 4, the angle of the arc represents the relative work for quality related to systems and software products.
importance of each quality characteristic to the product, ISO/IEC 25010 [
          <xref ref-type="bibr" rid="ref10">10</xref>
          ] defines both the quality-in-use and
and the length of the radius represents the achievement product quality model for systems and software products,
of product quality. For example, the figure shows usabil- while ISO/IEC 25012 [
          <xref ref-type="bibr" rid="ref11">11</xref>
          ] defines the data quality model.
ity is the most important quality of the target product, Quality-in-use represents the influence on
stakeholdand its achievement level is about 70 ers when the target entity is used under a certain context
of use, while product quality and data quality are the
capability of the target product and data themselves,
respectively to satisfy both stated and implies needs.
        </p>
      </sec>
      <sec id="sec-4-5">
        <title>5.2. Interpretation of the SQuaRE quality models in the quality state model</title>
        <p>Almost the same comparative review can be performed
for product quality requirements. By conducting the
above sprint review for each QiU requirement, it is
possible to evolve the quality requirements, making sure the
• Quality in Use [QiU]
– Efectiveness (magnitude of value created</p>
        <p>for the user)
– Eficiency (eficiency of user tasks)
– Satisfaction (positive impact on user’s</p>
        <p>mind &amp; attitude)
– Freedom of risk (avoidance of negative
im</p>
        <p>pacts)
– Usage Coverage (ability to perform
appropriately to anticipated/future usage
situations)
• Product quality characteristics for non-user use
(ease of development and maintenance work) [PQ
QiU ]
– Testability (ease of testing work)
– Analyzability (ease of failure analysis)
– Modifiability (ease of modification work)
– Installability (ease of installation work)
– Reusability (ease of reuse)</p>
      </sec>
      <sec id="sec-4-6">
        <title>5.3. True product quality</title>
        <p>The product quality characteristics other than the above
represent product-specific qualities and are classified as
below.</p>
        <p>• Quality of the functions that the system has
[PQ(Function), PQ(Set of Functions)]
– Functional accuracy
– Function appropriateness
– Time behavior
• Quality of a set of functions
– Functional completeness
– Usability (cognitive ease of use)
– Security
• Quality of the source code of the system
[PQ(Code)]</p>
        <p>– Modularity
• Characteristics of the system with respect to
possible failures [PQ(Failure)]
– Maturity (no failures ← few defects)
– Fault tolerance (service continuity against</p>
        <p>failures)
– Availability (service uptime against
fail</p>
        <p>
          ures)
– Recoverability (low loss against possible
failures)
addition, there are three patterns for the realization of
quality requirements: assignment to components,
functions, and architecture (structure and design guideline)
[
          <xref ref-type="bibr" rid="ref14">14</xref>
          ]. When A, B, and C are quality nodes (in 6), the rule
A→B | C means that A can be deployed into either B or
C.
        </p>
        <p>QiUFunction [with PQ(Function)]* | PQ(Set of
Functions)* | PQ(Outside)* | PQ(Failure)* PQ QiU
Function [with PQ(Function)]* | PQ(Code)* | Architecture
PQ(Function) DQ* PQ(Set of Functions), PQ(outside),
PQ(failure) Function [with PQ(Function)]* | DQ* |
Architecture</p>
        <p>QiU requirements can be deployed into Function and
Product quality requirements. The product quality
characteristics for non-user use are developed into functions,
code quality, and architecture. Architecture is embodied
in the components, their relationships and principles of
behavior, and design guidelines.</p>
        <p>When a system is decomposed into itscomponents,
the deployment of product qualityto them also occurs.
In this case, product quality can be inherited, not
inherited, or assigned among the components. In case of
the assignment, a product quality requirement such as
time eficiency and reliability, is decomposed into a set
of new requirements with diferent goals, each of which
assigned to each component. In case of including data
components, data quality (DQ) requirements are defined
for them.</p>
      </sec>
      <sec id="sec-4-7">
        <title>5.5. Quality evaluation of the SQuaRE quality model</title>
        <p>
          Quality evaluation in SQuaRE [
          <xref ref-type="bibr" rid="ref15">15</xref>
          ] selects important
quality characteristics and sub-characteristics, and defines
information needs for the target to measure them using
quality measures, as shown in 5 . The quality measure
• Relationship with outside of system [PQ(Out- quantifies the results of testing, user reviews, or static
side)] analysis of the target entity into one single value. The
– Capacity satisfiability value of the quality measure is mapped to a predefined
– Interoperability quality rating level to obtain a quality rating value.
        </p>
        <p>Quality DT considers non-functional testing, static
– Coexistence analysis, and user reviews conducted in each iteration
– Adaptability of agile development as quality activities, and integrates
– Accessibility the results of these activities to evaluate the achievement
– Resource eficiency of the corresponding quality requirements.</p>
      </sec>
      <sec id="sec-4-8">
        <title>5.4. Deployment of quality requirements in the SQuaRE quality requirements framework</title>
        <p>
          6 shows the meta-model of the quality state model. The
quality state model is realized as a directed acyclic graph
ISO/IEC 25030 Quality Requirements Framework [
          <xref ref-type="bibr" rid="ref12">12</xref>
          ], in which higher-level requirements are related to
lower[
          <xref ref-type="bibr" rid="ref13">13</xref>
          ] guides the flow of deployment of quality require- level realizations by the link ”supports”. A support link
ments through decomposition of the system. has an attribute of “contribution,” which indicates the
        </p>
        <p>Quality Digital Twin supports various deployment degree of support. The following node types are
considmethods of quality requirements by using patterns. In ered.</p>
      </sec>
      <sec id="sec-4-9">
        <title>5.6. Design of the quality state model</title>
        <p>• Quality requirement: QiU, PQQiU, PQ(Set of</p>
        <p>Functions), PQ(Outside), PQ(Failure),DQ
• Quality implementation: Function [with</p>
        <p>PQ(Function)], Architecture
• Quality activity: Functional testing,
Nonfunctional testing, Static analysis, User review</p>
        <p>Quality requirements can be supported by not only
quality requirements themselves, but quality
implementations and quality activities. Quality implementation
can be supported by Functional testing for Function,
Nonfunctional testing for Architecture, and Static analysis
for Architecture.</p>
      </sec>
      <sec id="sec-4-10">
        <title>5.7. Quality evaluation of the SQuaRE quality model</title>
        <p>In the Quality DT, quality management of agile
development is considered as a process of correctly evolving
the quality state model. In other words, while iteratively
developing the product, quality requirements (and their
relationships) are gradually established, and at the same
time, the degree of achievement of quality requirements
is measured from the results of quality activities in the
agile development, and corrective activities are
recommended for achieving the quality goals. This prevents
the agile development from deviating from the quality
goals.</p>
        <p>7 shows how the quality state model is used in a sprint.
At the sprint planning, the results of requirements
decomposition are input into it, the results of quality
activities are collected during executing the sprint, and at the
sprint review, quality evaluation are conducted on the
model and then the model is reviewed and, if necessary,
reorganized. The detailed usage flow is shown below.</p>
        <p>For all quality activities conducted up to this stage,
the quality evaluation value of quality node n i ,
achievement(n i ), is calculated using the following formula.</p>
        <p>where support(n i ) = n i1 , n i2 , ... , n im is the total
set of nodes supporting node n i , and contribution ij is
the contribution of node n ij to n i . In the evaluation
review of n i , the contents and quality evaluation values
of n i , all nodes supporting n i support(n i ) (including the
quality implementations and quality activities conducted
in this sprint), and the contribution ij of each node are
shown on the links, as shown in 8.</p>
        <p>In addition, the user can also check the degree to which
quality requirements have been met and the progress
made since the last time. By checking the correspondence
and contribution of the supporting nodes, it is possible to
review whether the implementation and activities truly
support the goal quality requirements and whether their
contribution is appropriate.</p>
      </sec>
      <sec id="sec-4-11">
        <title>5.8. Challenges in realizing Quality DT</title>
        <p>The system that realizes the quality digital twin consists
of three functions: a quality requirement decomposition
support function, a quality evaluation function, and a
quality dashboard function, as shown in 9.</p>
        <p>
          The followings need to be addressed for each function.
• Quality requirement decomposition function:
The description and decomposition of quality
requirements must be user-driven. It is necessary
to support appropriate decomposition without
dificulty for users.
• Quality evaluation function: It is necessary to
establish a method to link the results of
nonfunctional testing and static analysis to quality
evaluation. For static analysis, we plan to refer
to ISO/IEC 5055 [
          <xref ref-type="bibr" rid="ref16">16</xref>
          ].
• Quality dashboard function: what information
and how needed to displayed to help users take
the lead in quality management.
        </p>
      </sec>
    </sec>
    <sec id="sec-5">
      <title>6. Conclusion</title>
      <p>We proposed the concept of Quality Digital Twin, which
visualizes the quality of products in the agile
development, using the quality models of the ISO/IEC 25010.
This concept enables the realization of a system that
supports user-driven, high-quality agile development. The
feasibility of the concept is demonstrated by showing an
example of the structure and usage of the quality state
model, which is the core of the concept, as well as three
issues for the realization of the concept. We plan to
conduct prototyping and experiments to demonstrate the
efectiveness of the system.</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          [1]
          <string-name>
            <given-names>G.</given-names>
            <surname>Vial</surname>
          </string-name>
          ,
          <article-title>Understanding digital transformation: A review and a research agenda</article-title>
          ,
          <source>The Journal of Strategic Information Systems</source>
          <volume>28</volume>
          (
          <year>2019</year>
          )
          <fpage>118</fpage>
          -
          <lpage>144</lpage>
          . URL: https://www.sciencedirect.com/ science/article/pii/S0963868717302196. doi:https: //doi.org/10.1016/j.jsis.
          <year>2019</year>
          .
          <volume>01</volume>
          .003, sI: Review issue.
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          [2]
          <string-name>
            <given-names>T.</given-names>
            <surname>Komiyama</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Motoei</surname>
          </string-name>
          ,
          <article-title>Establishing international standards for systems and software quality requirements and evaluation, IWESQ@APSEC (</article-title>
          <year>2020</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          [3]
          <string-name>
            <surname>S. McConnell</surname>
          </string-name>
          ,
          <article-title>More efective Agile: A roadmap for software leaders</article-title>
          ., Construx Press,
          <year>2019</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          [4]
          <string-name>
            <given-names>B.</given-names>
            <surname>Meyer</surname>
          </string-name>
          , The Ugly,
          <article-title>the Hype and the Good: an assessment of the agile approach</article-title>
          ; Agile, Springer, Cham,
          <year>2014</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          [5]
          <string-name>
            <given-names>C.</given-names>
            <surname>Ebert</surname>
          </string-name>
          ,
          <string-name>
            <given-names>H. C.</given-names>
            <surname>Duarte</surname>
          </string-name>
          ,
          <article-title>Digital transformation</article-title>
          ,
          <source>IEEE Software 35</source>
          (
          <year>2018</year>
          ) 16
          <fpage>21</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          [6]
          <string-name>
            <given-names>K.</given-names>
            <surname>Kautz</surname>
          </string-name>
          ,
          <article-title>Participatory design activities and agile software development</article-title>
          ,
          <source>IFIP WG 8.2/8</source>
          .6 International Working Conference (
          <year>2010</year>
          )
          <fpage>303</fpage>
          -
          <lpage>316</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          [7]
          <string-name>
            <given-names>B. R.</given-names>
            <surname>Barricelli</surname>
          </string-name>
          ,
          <string-name>
            <given-names>E.</given-names>
            <surname>Casiraghi</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Fogli</surname>
          </string-name>
          ,
          <article-title>A survey on digital twin: definitions, characteristics, applications, and design implications</article-title>
          , IEEE access (????)
          <fpage>167653</fpage>
          -
          <lpage>167671</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          [8]
          <string-name>
            <given-names>K. Y. H.</given-names>
            <surname>Lim</surname>
          </string-name>
          ,
          <string-name>
            <given-names>P.</given-names>
            <surname>Zheng</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C.-H.</given-names>
            <surname>Chen</surname>
          </string-name>
          ,
          <article-title>A state-of-theart survey of Digital Twin: techniques, engineering product lifecycle management and business innovation perspectives</article-title>
          ,
          <source>Journal of Intelligent Manufacturing</source>
          <volume>31</volume>
          (
          <year>2020</year>
          )
          <fpage>1313</fpage>
          -
          <lpage>1337</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          [9]
          <string-name>
            <given-names>F.</given-names>
            <surname>Tao</surname>
          </string-name>
          , et al. ,
          <article-title>Digital twin in industry: State-of-theart</article-title>
          ,
          <source>IEEE Transactions on Industrial Informatics 15.4</source>
          (
          <year>2018</year>
          )
          <fpage>2405</fpage>
          -
          <lpage>2415</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          [10] ISO/IEC 25010:
          <year>2011</year>
          ”
          <article-title>Systems and software engineering - Systems and software Quality Requirements and Evaluation (SQuaRE) - System and software quality models</article-title>
          .
          <source>”</source>
          ,
          <year>2011</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          [11] ISO/IEC 25012:
          <year>2008</year>
          <article-title>Software engineering - Software product Quality Requirements and Evaluation (SQuaRE) - Data quality model</article-title>
          ,
          <year>2008</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          [12] ISO/IEC 25030:
          <article-title>2019 Systems and software engineering - Systems and software quality requirements and evaluation (SQuaRE) - Quality requirements framework</article-title>
          ,
          <year>2019</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          [13]
          <string-name>
            <given-names>T.</given-names>
            <surname>Nakajima</surname>
          </string-name>
          , T. Komiyama,
          <article-title>Applying Quality Requirements Framework to an IoT System and</article-title>
          its Evaluation,
          <source>International Journal on Advances in Internet Technology</source>
          <volume>12</volume>
          (
          <year>2019</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          [14]
          <string-name>
            <given-names>P.</given-names>
            <surname>Clements</surname>
          </string-name>
          , et al.,
          <article-title>Documenting software architectures: views and beyond (2nd edition)</article-title>
          ,
          <source>AddisonWesley Professional</source>
          ,
          <year>2010</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref15">
        <mixed-citation>
          [15]
          <string-name>
            <given-names>T.</given-names>
            <surname>Nakajima</surname>
          </string-name>
          ,
          <article-title>About the Framework of Quality Evaluation Using SQuaRE, APSEC (</article-title>
          <year>2020</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref16">
        <mixed-citation>
          <source>[16] ISO/IEC 5055 Information technology-Software measurement-Software quality measurementAutomated source code quality measures</source>
          ,
          <year>2021</year>
          .
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>