=Paper=
{{Paper
|id=Vol-3612/IWESQ_2023_Paper_05
|storemode=property
|title=Research on Software Quality Characteristics based on MBSE System
|pdfUrl=https://ceur-ws.org/Vol-3612/IWESQ_2023_Paper_05.pdf
|volume=Vol-3612
|authors=Jiangtao Che,Yangyang Zhang,TWenpeng Li,Jianxun Guo
|dblpUrl=https://dblp.org/rec/conf/apsec/CheZG23
}}
==Research on Software Quality Characteristics based on MBSE System==
https://ceur-ws.org/Vol-3612/IWESQ_2023_Paper_05.pdf
Research on Software Quality Characteristics based
on MBSE System
Jiangtao Che, Yangyang Zhang*, Wenpeng Li, Jianxun Guo
China Electronics Standardization Institute, Beijing, China
Abstract
MBSE (Model-based Systems Engineering) is an engineering method that uses graphical models to
describe and design complex systems, which has the advantages of unambiguous knowledge
representation, reusability of models and integration of system design. Based on MBSE, through
modeling and simulation, the quality characteristics of product quality such as functionality, reliability
and testability are improved. The important significance of MBSE is that it provides new ideas and
methods for the software engineering field, helps to improve the quality and development efficiency of
software products, and has broad development prospects in the software engineering field.
Keywords
MBSE, software quality, product quality mode, quality standard
1
Engineering Vision 2020 as "Model-based Systems
1. Introduction Engineering [MBSE] is a paradigm that uses formalized
representations of systems, known as models, to
In the early stage of system engineering, the support and facilitate the performance of Systems
information generated by the system is described and Engineering [SE] tasks throughout a system’s life
recorded in the form of documents. However, with the cycle."[2] INCOSE stressed that MBSE is the
continuous improvement of the scale and complexity development trend of future systems engineering
of the system, the shortcomings and deficiencies of the methods and technologies, and a change in the field of
traditional document-based system design method are systems engineering, and proposed the MBSE vision
becoming more and more prominent, such as plan for the first time at the conference, and planned to
inaccurate information representation, easy to realize the gradual maturity of MBSE theory and
generate ambiguity, difficult to find the required practice system from 2007 to 2020, which represents
information from massive documents, and unable to that MBSE will become an important development
connect with other engineering designs[1]. direction of systems engineering in the future.
With the rapid development of computer and MBSE has been recognized by many experts in the
information technology and engineering technology in field of systems engineering, and is becoming the
various fields, it is becoming easier and easier to foundation of complex system design. Compared with
describe the system with object-oriented, graphical traditional document-based system engineering
and visual system modeling language, and the methods, MBSE models become the core of complex
application proportion of model in the system system design from the requirements analysis stage to
development work is also increasing. Model based the evaluation stage. MBSE uses digital modeling
systems engineering (MBSE) came into being. MBSE instead of writing documents for system scheme
method can effectively solve the problems of design, and converts all the nouns, verbs, adjectives
document-based system engineering methods in and parameters describing system structure, function,
parameter acquisition and technical state performance and specification requirements in design
management, and is a powerful tool to effectively deal documents into digital model expressions. The system
with system complexity. design of complex system is realized through the
In 2007, the International Council on Systems continuous evolution and iterative increase of the
Engineering (INCOSE) defined MBSE in its Systems model, which has the advantages of unambiguous
knowledge expression, reusability of the model,
5th International Workshop on Experience with SQuaRE Series andits
Future Direction, December 04, 2023, Seoul, Korea
chejt@cesi.cn (J. Che); zhangyy@cesi.cn (Y. Zhang)
0000-0002-5456-1538 (J. Che); 0009-0006-4940-8527(Y. Zhang)
© 2023 Copyright for this paper by its authors. The use permitted under Creative
Commons License Attribution 4.0 International (CC BY 4.0).
CEUR Workshop Proceedings (CEUR-WS.org)
CEUR
ceur-ws.org
Workshop ISSN 1613-0073
Proceedings
29
integration of system design, etc., and can well solve but also reduce development costs and
the problems and challenges brought by the increase improve the quality and reliability of the
of system complexity. system.
Compared with TSE(Traditional Systems ⚫ Reliability: By building digital models, MBSE
Engineering), the characteristics of MBSE are reflected can conduct comprehensive simulation and
in three aspects: modeling language, modeling ideas testing of the system to identify and resolve
and modeling tools. The main difference at the system potential problems and risks. This
modeling level is that the integration degree and simulation can be carried out before the
executable degree of the system model are greatly
actual system is built, thus avoiding
improved, as shown in Table 1.
problems in actual operation and improving
Table 1 the reliability and stability of the system.
Characteristics of TSE and MBSE
⚫ Security: MBSE uses a modeling approach to
TSE MBSE design and analyze the system, which can
Modeling Natural modeling System modeling more comprehensively consider the security
language language language requirements of the system, integrate the
The main line is The main line is the security concept from the beginning of the
Modeling the function layer design, and reduce potential security risks.
method decomposition of decomposition of At the same time, MBSE can detect and solve
the system the system entity potential security problems in advance
Computer (with through simulation and testing. Compared
Computer (with with traditional system testing methods,
system modeling
natural language MBSE can test system security more
language editing
Modeling editing software
software installed), comprehensively and systematically.
tool installed), mainly
mainly to process Combined with the characteristics of MBSE system,
for processing
text
graphical, visual the mapping relationship between MSBE system and
and formal models product quality characteristics can be established, as
It is composed of shown in Fig. 1.[3]
Stored in the
"discrete"
computer's model
System reports, which
library and
model rely on manual
automatically
and manual
correlated
correlation
2. Quality Characteristics
of MBSE Systems
Compared with traditional software systems, the
most important difference between MBSE based
software systems is the formal modeling of the
technical process throughout the whole life cycle, Figure 1: Relationship between MBSE system and
focusing on the formalization rather than the existence product quality model
of modeling. Formal modeling has the following
advantages:
⚫ Unambiguous knowledge expression: MBSE 3. MBSE System and
describes the structure and behavior of the
system using digital models that are based on
Software Quality Model
clear rules and standardized symbols and
terms, avoiding the semantic ambiguity or The model quality measurement includes two
parts: the quality model and the quality measurement.
misunderstanding that can occur in
The quality model gives the quality characteristics
traditional documents. Through digital
from which the quality of the model should be
models, MBSE can provide clear, consistent considered, and the quality measurement gives the
and accurate information that enables the quantification way of each quality characteristic.
design and functionality of the system to be Based on the software product quality model in Fig. 1,
understood and shared by all concerned, combined with the characteristics of MBSE system, the
thereby improving the functional model-based quality measurement m is formed after
correctness of the system design. tailoring, as shown in Fig. 2.
⚫ Reusability: The model in MBSE is
independent of a specific project or system
and can be reused in different projects or
systems, which can not only improve work
efficiency and reduce duplication of labor,
30
The quality measures of applicability would
Model-based quality
include:
measurement
The number of modeling languages used mainly
evaluates the applicability of military software in
Correctness Effectiveness Applicability Security Reliability Testability model selection;
Average Proportion of
requirements
Number of
Reliable
number of
Model size
Model complexity is mainly measured by the
component
error rate in the model
modeling
languages
Requirement
model
systems/
components
and number of nodes/elements in the model, the number
of links in the model, the number of inbound nodes and
complexity
used traceability
Test cases/
Individual Reliability
the number of outbound nodes.
scripts Reusability
component Model Courier quantitative
generated by of model
error rate complexity consistency simulation
model elements
evaluation
D. Security
To build a complete model system, first of all, we
need to analyze the needs of users, establish the
Figure 2: Model-based quality measurement model
original demand and demand model, and establish the
traceability relationship between them; Then the
A. Correctness functional model and architecture model are modeled
The main purpose of this work is to verify the according to the requirements, and the traceability
consistency of the model with standards or modeling relationship between requirements and design is
specifications. The model-based software test model is established. Finally, the system model after iteration is
consistent with the rules of the formal description verified and evaluated.
(modeling language, modeling guide), so that test In the system development based on MBSE,
cases or test data can be generated according to the requirement analysis starts from the top-level
model-based software test model without problems requirement to model the requirements layer by layer,
due to incorrect syntax. and establishes the requirement traceability
The quality measure of correctness is divided into relationship through the requirement model. The
average component error rate and individual purpose of requirements analysis is to obtain the
component error rate. The average component error system requirements from the original requirements,
rate can be used to assess the overall situation quality so as to support the system architecture design.[4]
of the entire software system by the number of After completing the design of the requirement
modeling requirements that are wrong/the number of model, it is necessary to carry out functional analysis
all requirements in the model; The error rate of a in the top-level design process. Analyze and model
single component can be measured by the number of each functional component, design the realization
incorrect modeling requirements in the concerned method of the system use case through the use case
software configuration item/the number of all model, build the functional architecture, and establish
requirements in the concerned software configuration the traceability relationship between requirements
item, which is mainly to assess the model correctness and design. Function analysis reflects the system's
of a single component in the software system. activity and activity execution process by drawing
B. Effectiveness activity diagram, and reflects the system's
requirements. Moreover, through the corresponding
The main purpose of this work is to confirm the relationship between functions and system
consistency between the model and user requirements requirements in the activity diagram, the functional
and construction objectives. The content of the model model can also be linked to its corresponding
is correct and expresses what needs to be expressed. requirements to complete the establishment of the
The resulting product is "useful", the test script can be traceability relationship between requirements and
executed, the test case can be executed by the test design.
engineer, and no errors can occur due to incorrect test
cases. Finally, on the basis of the functional model, the
overall route and technical framework of the system
Quality measures of effectiveness are to include: are analyzed, and the system architecture model is
The proportion of requirements in the model = the designed. Architecture design is the transition from
number of requirements in the model to complete the top layer of the system to the software
modeling/the total number of requirements in the implementation layer. Through module definition, the
requirement specification; relationship between modules is shown, the
components and characteristics of the system are
Test cases/scripts generated by model = Number described, and the design and development of the
of test cases or scripts generated by the system model is guided. The architecture design
model/number of requirements modeled in the model. facilitates the refinement of the black box model and
thus the conversion to the white box model of the
C. Applicability module. Through architecture design, the
The main purpose of this work is to verify the requirements and functions of the system are assigned
consistency of the model with the test objectives and to components, ensuring the consistency of
confirm the consistency with the test generation information transmission in the design process and
mechanism. Only when the model-based software test realizing the accurate expression of the top-level
model is appropriate for the specified test objectives framework of the system.
and the specified test generation mechanism can the
products derived from the model-based software test E. Reliability
model fully meet the expectations.
31
Complex engineering systems are generally costly The opportunities MBSE brings to system
and high risk systems. If a large number of system testability are mainly reflected in the modeling and
reliability design problems can only be found after the sustainable simulation verification of the system
physical construction integration, its change costs and design itself, which enables the virtual injection of
expenses are inevitably unacceptable. Therefore, faults into the system model to investigate their impact
reliability design verification before construction is and evaluate the pros and cons of the test scheme, thus
necessary for complex systems. However, at the same achieving a more objective, comprehensive and
time, the reliability design verification of complex accurate testability application.
systems is very difficult, which is more difficult than
the functional performance verification of the system. Test requirement is the basic basis for designing
The reliability verification of the system should in-machine test and automatic test equipment, which
consider the impact of many variables such as the is conducive to achieving the consistency of testability
system's use scenario, environmental conditions, analysis, automatic test and fault diagnosis.[6]
functional requirements, physical composition and Automating the testability analysis and design
personnel operation, which has long been a difficult process based on MBSE requires solving the following
problem in engineering.[5] key technologies.
In the component/system integration verification ⚫ Complex system multi-level co-simulation
stage, the reliability level of components and systems verification technology. This is not only the
is virtual verified by means of component level and core of future testability technology, but also
system functional reliability simulation analysis to a prerequisite for high-quality MBSE. The
determine whether they meet the requirements. This
hardware and software mixed simulation
technical link needs a set of system reliability
implementation and cooperation and other
simulation verification method based on multi-
contents support the simulation verification
physical model support, comprehensive use of all
levels of the system, multi-domain physical simulation of system functions and performance.
model, high fidelity simulation verification of ⚫ Fault data acquisition technology based on
reliability. automatic fault injection and simulation.
Relying on the high-precision integrated Automatic batch injection of faults in
simulation model provided by MBSE, the multi- simulable models is used to ensure the
physical simulation model close to the real use coverage of real underlying faults. One of the
scenario and environmental conditions is built early, key technical points is the transformation of
and the system reliability is fully verified before the low-level physical faults to high-level fault
physical construction, and the weak link of the system behavior descriptions, which makes it
reliability design is improved. possible to implement credible fault
behavior simulation at a higher level of
The key technical aspects of system reliability
simulation and verification process based on multi- abstraction (such as behavior level), reduce
physical model include: the dependence on design details, and
improve simulation performance and
(1) Fault modeling of system components operability.[7]
Based on the multi-physical regional energy model ⚫ Data-driven testability modeling, analysis
of the system, various types of component fault and design techniques. Based on the above
simulation modeling are carried out, including fault simulation data, feature extraction is
mutation fault modeling, performance degradation carried out to obtain fault samples, and then
fault modeling, logic error fault modeling, etc. the intelligent classification algorithm is
(2) Formalized requirement definition and studied according to the characteristics of
verification the samples. Ideally, if the sample size is
sufficient and balanced, a cross-validated
This paper mainly uses linear temporal logic and testability analysis method can be used to
other formal languages to define reliability
quantitatively predict key testability
requirements by strict predicate and combinatorial
indicators. Testability design is to optimize
logic, and uses model simulation to automatically
output counterexamples of violating requirements, so fault features according to optimization
as to verify whether the qualitative requirements of objectives such as the number of test points
reliability are met. or test cost among many candidate
features.[8]
(3) Reliability quantitative simulation evaluation
Metrics and key performance indicators that may
In the system reliability simulation evaluation, be monitored include:
there are usually two technical approaches. One is
based on the fault mode related data of the system ⚫ Manages and tracks the number of
components, using Monte Carlo sampling to evaluate requirements in the MBT (Model-Based
the reliability. One is to use scenario model, operation Testing)model and the percentage of
model, fault mechanism model and multi-physical requirements covered by the test cases.
model to achieve quantitative evaluation of key
component reliability and system functional reliability ⚫ Size and complexity of the MBT model.
by task scheduling simulation.
F. Testability
32
⚫ The number of test cases and scripts [7] TANG X F, XU A Q, LI Ｒ F, et al． Simulation-
generated, and the number of test based diagnostic model for automatic testability
cases/scripts produced per person per day. analysis of analog circuits[J]. IEEE Transactions
on Computer-Aided Design of Integrated Circuits
⚫ Number of requirement defects found in
MBT modeling activities. and System, 2018, 37(7) : 1483 - 1493．
[8] TANG X F, XU A Q, NIU S C. KKCV-GA-based
⚫ The reusability of MBT model elements from method for optimal analog test point selection[J].
one project to another. IEEE Transactions on Instrumentation and
Measurement,2017, 66( 1) : 24 - 32
⚫ The extent to which the MBT model is used
by the project stakeholders
(business/development/test).
⚫ Percentage increase in efficiency versus
productivity of previous test design methods
(lower cost testing).
⚫ Percentage improvement in efficiency,
defect finding rate compared to previous test
design methods (better testing).
4. Conclusion
MBSE has a broad vision for improving the quality
of software products. As the technology continues to
evolve, MBSE methods and tools will continue to be
refined to more accurately describe and simulate
complex systems, further improving the functionality,
reliability and maintainability of software products. At
the same time, MBSE will be combined with new
technologies such as artificial intelligence and big data
to achieve more intelligent software design and quality
management, and improve the quality and
development efficiency of software products. In the
future, MBSE is expected to become an important
development trend in the field of software engineering
and make greater contributions to the improvement of
software product quality.
References
[1] Thomas A J, Jonathan M J, Paredis C J J. et al.
Modeling Continuous System Dynamics in SysML
[C]. Proceeding of the International Mechanics
Conference and Exhibition. Seattle: 2007: 1-11.
[2] International Council on Systems Engineering
(INCOSE). Systems engineering vision 2020[R].
Seattle, WA: International Council on Systems
Engineering, 2007: 5-32.
[3] ISO/IEC 25010:2023. Systems and software
engineering Systems and software Quality
Requirements and Evaluation (SQuaRE) Product
quality model.
[4] BAJAJ M, BACKHAUS J, WALDEN T, et al.
Graphbased digital blueprint for model based
engineering of complex systems[J]. INCOSE
International Symposium, 2017, 27(1): 151-169.
[5] WU J, YAN S, ZUO M J. Evaluating the reliability of
multi-body mechanisms: A method considering
the uncertainties of dynamic performance[J].
Reliability Engineering & System Safety, 2016,
149:96-106.
[6] Cook D, Schindel W D. Utilizing MBSE patterns to
accelerate system verification[J]. Insight, 2017,
20(1): 32-41.
33