=Paper=
{{Paper
|id=Vol-2145/p01
|storemode=property
|title=An Approach: SysML-based Automated Consistency Evaluation of the System Requirements Specification
|pdfUrl=https://ceur-ws.org/Vol-2145/p01.pdf
|volume=Vol-2145
|authors=Jovita Bankauskaite,Aurelijus Morkevicius
}}
==An Approach: SysML-based Automated Consistency Evaluation of the System Requirements Specification==
https://ceur-ws.org/Vol-2145/p01.pdf
An Approach: SysML-based Automated Consistency
Evaluation of the System Requirements Specification
Jovita Bankauskaite
Aurelijus Morkevicius
Department of Information Systems
Department of Information Systems
Kaunas University of Technology
Kaunas University of Technology
Kaunas, Lithuania
Kaunas, Lithuania
jovita.bankauskaite@ktu.lt
aurelijus.morkevicius@ktu.lt
Abstract— Systems Modeling Language (SysML) is used for study, poor requirements management is the second most
specifying, analyzing, designing and verifying complex systems, common reason for project failure [4].
and is designed to provide simple but powerful constructs for
modeling a wide range of Systems. SysML is not a methodology, In order to reduce the risk of mistakes detection and
nor a method and there are thousands of different ways to describe correction in the late stage of development, it is desirable and
the using it. In this case, there cannot be a single, universal important to identify the inconsistencies in a requirements
approach to evaluate the consistency of the requirements specification in the early stages of systems engineering. The
specification. It is necessary to choose a specific method in mistakes due to incompleteness, inconsistency, and ambiguity
combination with SysML to accurately and comprehensively introduced at the stage of requirements engineering are difficult
evaluate the consistency of requirements specification. The and more expensive to correct than those introduced in later
consistency evaluation of requirements specification in model- stages of system development [5]. Mistakes in requirements
based system engineering (MBSE), depending on the modeling specification may arise if the consistency of the specification is
language and method is quite a new practice. This opens up violated or the stakeholder requirements are misrepresented by
discussions of how to utilize SysML provided infrastructure to the specification. Completeness and correctness (C&C) analysis
evaluate the System Requirements Specification (SRS) and of requirements specification aims to eliminate occurred
achieve a high-quality of the SRS. In this paper, a new approach mistakes.
of how requirements specification, expressed with sufficient
precision in SysML can be used for automated consistency In this paper, we focus on a subset of the C&C task –
evaluation. correctness analysis only. We understand the correctness of SRS
as the nonexistence of inappropriate relationships between
Keywords—SysML, MBSE Grid, Consistency Metrics, System requirements and model elements. The question is how to utilize
Requirements Specification, Requirements Engineering, MBSE SysML provided infrastructure to successfully achieve a high
quality of the requirements specification: what method to use in
I. INTRODUCTION
combination with SysML.
Due to model-based engineering progress in recent years,
system engineering slowly but surely moves from document- In this paper, we propose a new approach of how
based system engineering to model-based system engineering. requirements specification that is expressed in SysML in
Nowadays, MBSE is enabled by Systems Modeling Language. combination with MBSE Grid method can be used for
automated consistency evaluation of the system requirements
SysML is a general-purpose graphical modeling language specification.
that supports the analysis, specification, design, verification, and
validation of complex systems. The language is intended to The MBSE Grid method guides how to specify principal
create cohesive and consistent models of structure, behavior areas of the system model and how to manage different layers of
including their interconnections. SysML introduces requirement abstraction [7].
diagrams, which represent requirements and their relationships The MBSE Grid is organized in a matrix view. Columns
to other requirements, design elements and test cases [1]. represent four main aspects of systems engineering
Requirements engineering is one of the most important and (requirements, system structure, system behavior and
critical phases in MBSE which consists of two main processes: parameters). Rows represent two main viewpoints: one to define
specification and management. Generally, for systems engineers the problem in order to understand it, other to provide one or
are more important requirements specification process than several alternative solutions to solve it. Cells of the grid (Fig. 1)
requirements management process. While managers focus is on represent different views of model-based systems engineering
the requirements management process, but they have a poor [8]. Specified traceability among view specifications is a very
understanding of the benefits of MBSE. According to PMI's important aspect of the MBSE Grid method. The method helps
to organize and maintain the model.
Copyright held by the author(s).
1
This research is carried out using MagicDraw toolset, which In conclusion, all the analyzed methods to evaluate the
supports SysML. It was chosen because of several published consistency of requirements specification encounter several
studies, e.g. [9], [2], [10], [11]. common issues: (i) unclear traceability relationships between
requirements and design elements, (ii) unsupported consistency
The rest of this paper is structured as follows: in section 2, evaluation at all stages of the requirements specification, (iii) it
the related works are analyzed; in section 3, the proposed is difficult to interpret the results of consistency evaluation.
approach for automated consistency evaluation of the
requirements specification is presented; in section 4, evaluation Overall, researches carried out in this area have very little
of the proposed approach is described; in section 5, the achieved proof of its successful application on real-industry cases or is
results, conclusions, and future work directions are indicated. very specific to a small area of application and specific tools
dependent. We are proposing a more generic, easy to use
II. RELATED WORKS approach, applicable to the majority of SysML modeling tools
There is a large number of research papers on the consistency for different systems engineering domains. The proposed
analysis of requirements specification. Most of them are applied approach in combination with MBSE Grid will evaluate the
to the small area of the domain or a specific tool, e.g. [12], [13], consistency of each stage of requirements specification. This
[3]. will help to monitor the quality of SRS and make the necessary
decisions in the early stage of requirements specification
Several authors proposed methodologies for evaluation of process.
consistency within the UML models which are applicable to
SysML, as SysML is the extension of UML. Methods defined in III. AN APPROACH FOR CONSISTENCY EVALUATION OF SYSTEM
[14], [15], [16], [17], [18] use formal techniques for consistency REQUIREMENTS SPECIFICATIONS
evaluation, e.g. Object-Z in [14], algebra in [15], attributed
This section describes the proposed approach in detail.
graph grammars in [16] focusing mainly on class diagrams and
behavior diagrams. [17] describes an algorithmic approach for The approach consists of the following metric groups that are
consistency evaluation between UML sequence and state defined on the basis of the principles of MBSE Grid method:
machine diagrams while [18] proposes a declarative approach
using process algebra CSP for consistency evaluation between A. Requirements Refinement Metrics
sequence and state machine diagrams. B. Requirements Satisfaction Metrics
C. Requirements Verification Metric
The metric groups mentioned above compute only atomic
model elements that are linked to the atomic requirements (child
requirement). The relation between atomic requirements and
atomic model elements eliminates the ambiguities that may
occur having relations between higher level elements.
Fig. 1. MBSE Grid
Use of traceability relationships to evaluate the consistency
of the requirements specification has been defined in [19], [20],
[21], [22]. [20] proposes consistency analysis method to identify
the inconsistencies in the requirements. This method checks
requirements consistency in forward and backward directions.
The inconsistencies found between requirements and structural
Fig. 2. MBSE Grid Traceability
elements are logged into configuration inconsistency matrix. A
method in [22] uses traceability and manages fuzzy relationships
between high-level software artifacts (requirements), uses case The approach is based on specified traceability relationships
models and black box test plans. in MBSE Grid [Fig. 2]. Requirements Refinement Metrics
compute the consistency of requirements specification at
In [23] publication is proposed set of metrics based on problem layer. This metrics group evaluates the refinement of
requirements and UML design models for an object-oriented stakeholder needs by elements that are specified at stages of
system to measure the degree of consistency of design models functional analysis, logical subsystems communication and
with respect to requirements. The metrics defined in this method measurements of effectiveness of Subsystems (MoES).
are based on the linking of two different types of elements, e. g. Requirements Satisfaction Metrics compute the consistency of
class and activity. requirements specification at solution layer. This metrics group
2
evaluates system requirements satisfaction by elements that are SE – quantity of structure elements defined in the logical subsystem
specified at stages of component behavior, structure and communication analysis
parameters. Requirements Verification Metric evaluates the Interface Requirements Refinement by Proxy Elements
consistency between system requirements and test cases. metric
The proposed method concerns the consistency evaluation of This metric evaluates the utilization of proxy elements to
the system requirements specifications. An approach is refine requirements. Proxy ports defined in the logical
implemented in the MagicDraw modeling tool. subsystem communication have to refine the atomic
In order to obtain the more precise evaluation results of interface requirements of stakeholder. Below is provided the
requirements specification, metrics are categorized by three metric formula.
aspects of system engineering: Behavior, Structure, and 𝑃𝑃𝐸𝑅𝐼𝑅
Parameters. 𝑀𝐸𝑅𝑃𝑃𝐸 = × 100%
𝑃𝑃𝐸
The following subsections describe in detail each
consistency metric of requirements specification. MERPPE – interface requirements refinement by proxy elements
metric
A. Requirements Refinement Metrics
PPERIR – quantity of proxy port elements used to refine interface
This metric group evaluates the consistent use of model requirements
elements to refine stakeholder needs. The stakeholder needs are
PPE – quantity of proxy port elements defined in the logical
refined by the behavior elements specified in the functional subsystem communication analysis
analysis, by the structure elements specified in the logical
subsystem communication and by the parameters specified at the Performance Requirements Refinement by Parameters
measurements of effectiveness. Elements metric
The metric group of requirements refinement consists of the This metric evaluates the utilization of value property
following metrics: elements to refine requirements. The value property
elements defined in the measurements of effectiveness have
Functional Requirements Refinement by Behavior to refine the atomic performance requirements of
Elements Metric stakeholder. Below is provided the metric formula.
This metric evaluates the utilization of behavior elements to 𝑃𝐸𝑅𝑃𝑅
refine requirements. The atomic activity elements defined in 𝑀𝐸𝑅𝑃𝐸 = × 100%
the functional analysis have to refine the atomic functional 𝑃𝐸
requirements of stakeholder. Below is provided the metric
MERPE - performance requirements refinement by parameters
formula. elements metric
𝐵𝐸𝑅𝐹𝑅
𝑀𝐸𝑅𝐵𝐸 = × 100% PERPR – quantity of parameters elements used to refine performance
𝐵𝐸
requirements
MERBE - functional requirements refinement by behavior elements PE – quantity of behavior elements defined in the measurements of
metric effectiveness analysis
BERFR – quantity of behavior elements used to refine functional B. Requirements Satisfaction Metrics
requirements This metric group evaluates the consistent use of model
BE – quantity of behavior elements defined in the functional elements to satisfy system requirements.
analysis
The metric group of model elements usage for requirements
Physical Requirements Refinement by Structure satisfaction consists of the following metrics:
Elements metric
Functional Requirements Satisfaction by Behavior
This metric evaluates the utilization of structure elements to Elements metric
refine requirements. The atomic block or part elements
defined in the logical subsystem communication have to This metric evaluates the utilization of behavior elements to
refine the atomic physical requirements of stakeholder. satisfy requirements. The atomic activity elements defined
Below is provided the metric formula. in the component behavior have to satisfy the atomic
functional requirements of the system. Below is provided the
𝑆𝐸𝑅𝑃ℎ𝑅 metric formula.
𝑀𝐸𝑅𝑆𝐸 = × 100%
𝑆𝐸 𝐵𝐸𝑆𝐹𝑅
𝑀𝐸𝑆𝐵𝐸 = × 100%
𝐵𝐸
MERSE - physical requirements refinement by structure elements
metric
MESBE - functional requirements satisfaction by behavior elements
SERPhR – quantity of structure elements used to refine physical metric
requirements
3
BESFR – quantity of behavior elements used to satisfy functional C. Systems Requirements Verification metrics
requirements
This metric evaluates the consistent use of test cases to
BE – quantity of behavior elements defined in the component verify the system requirements. Defined test cases have to
behavior analysis verify atomic system requirements. Below is provided the
Physical Requirements Satisfaction by Structure metric formula.
Elements metric 𝑇𝐶𝑆𝑅
𝑇𝐶𝑉 = × 100%
The metric evaluates the utilization of structure elements to 𝑇𝐶
satisfy requirements. The atomic block or part elements
defined in the component assembly have to satisfy the TCV – system requirements verification metric
atomic physical requirements of the system. Below is
provided the metric formula. TCSR – quantity of test cases used to verify system requirements
𝑆𝐸𝑆𝑃ℎ𝑅 TC – quantity of test cases
𝑀𝐸𝑆𝑆𝐸 = × 100% D. Refinement Evaluation of Stakeholder Needs
𝑆𝐸
This subsection describes in detail the principles of the
MESSE - physical requirements satisfaction by structure elements refinement evaluation of stakeholder needs applying the
metric
requirements refinement by behavior elements metric (1).
SESPhR – quantity of structure elements used to satisfy physical
requirements The figure below (Fig. 3) represents the stakeholder
needs refinement by atomic activity element.
SE – quantity of structure elements defined in the component
assembly analysis
Interface Requirements Satisfaction by Proxy Elements
metric
This metric evaluates the utilization of proxy elements to
satisfy requirements. Proxy ports defined in the component
assembly have to satisfy the atomic interface requirements
of the system. Below is provided the metric formula.
𝑃𝑃𝐸𝑆𝐼𝑅
𝑀𝐸𝑆𝑃𝑃𝐸 = × 100%
𝑃𝑃𝐸
MESPPE – Interface Requirements Satisfaction by Proxy Elements
metric
PPESIR – quantity of proxy port elements used to satisfy functional
requirements
PPE – quantity of proxy port elements defined in the component
assembly analysis
Performance Requirements Satisfaction by Parameters Fig. 3. Refinements of stakeholder requirements
Elements metric
The metric evaluates the utilization of value property First, we calculate the quantity of atomic activity
elements to satisfy requirements. The value property elements that are defined in the functional analysis.
elements defined in the component parameters analysis have Activities have to represent the behavior of the system.
to satisfy the atomic performance requirements of the Second, we calculate the quantity of atomic activity
system. Below is provided the metric formula. elements that are used to refine the atomic functional
𝑃𝐸𝑆𝑃𝑅 requirements of stakeholder.
𝑀𝐸𝑆𝑃𝐸 = × 100% Third, we calculate the evaluation of requirement
𝑃𝐸
refinement by behavior elements using the particular metric.
MESPE - performance requirements satisfaction by parameters Below is provided the result of the evaluation of
elements metric
stakeholder need refinement according to Fig. 3.
PESPR – quantity of parameters elements used to satisfy functional
requirements BERFR = 1
PE – quantity of parameters elements defined in the component BE = 2
parameters analysis 1
𝑀𝐸𝑅𝐵𝐸 = × 100% = 50%
2
4
This indicates that 50% of the activities which are started the refinement of performance requirements stage. When
specified at the stage of function analysis are used to refine all metrics reached over 90%, it was decided that the refinement
the stakeholder needs. of requirements specification is sufficient.
IV. CASE STUDY
This section describes the case study of the proposed
approach. This is a case study of a commercial project to
evaluate the consistency of the requirements specification.
The following is a detailed description of the consistency
analysis of requirements specification. The commercial project
is based on SysML and is modeled in the MagicDraw toolset.
The modeling carried out in accordance with the principles of
MBSE grid.
Requirements specifications Consistency metrics have been
computed over the entire period of SRS. After each metric
calculation, the responsible persons have been analyzed the
metrics data and made appropriate decisions to ensure a high
quality of the SRS. Fig. 6. Requirements Satisfaction Diagram
Fig. 4 shows the part of requirements satisfaction metric
table that is computed in the MagicDraw tool. For effective In Fig. 6 is displayed satisfaction analysis diagram of
analysis, metrics data was exported to the excel and the visual requirements specifications over a period specifying the solution
charts were created according to the metrics data. layer of requirements specification. First, the functional
requirements of the system have been satisfied by behavior
elements. Reaching the 82% of behavior elements usage for
satisfying the functional requirements of the system has been
started another stage, the satisfaction of physical and interface
requirements. Reaching over the 85% of proxy ports usage for
satisfying the interface requirements and structure elements
Fig. 4. Metric Table usage for satisfying the physical requirements has been started
other stage, the satisfaction of performance requirements. When
Below is provided a detailed analysis of each metric groups all metrics reached over 90%, it was decided that the satisfaction
that are presented in the charts. of requirements specification is sufficient.
V. CONCLUSION AND FUTURE WORKS
The analysis of existing consistency evaluations methods for
the requirements specification disclosed that there are multiple
different methods available. The majority of them cannot be
used in combination with systems modeling techniques, such as
SysML, in practice. We found a need to propose a more generic,
easy to use approach, applicable to the majority of SysML
modeling tools for different system engineering domains.
In this paper, we proposed a new approach of how
requirements specification, expressed with sufficient precision
in SysML, can be used for automated consistency evaluation.
The approach consists three metric groups that are defined on
the basis of the principles of MBSE Grid method: Requirements
Fig. 5. Requirements Refinement Diagram
Refinement Metrics, Requirements Satisfaction Metrics,
Requirements Verification Metric.
In Fig. 5 is displayed refinement analysis diagram of
requirements specification. Requirements refinement metrics We have implemented the proposed approach in the
have been computed over a period specifying the problem layer MagicDraw CASE tool and demonstrated an example case
of requirements specification. First, the functional requirements study. After analyzing the case study, it was determined that
of stakeholder have been refined by behavior elements. calculation of the consistency metrics over a period contributes
Reaching the 85% of behavior elements usage for refining to ensure a high quality of each stage of requirements
functional requirements of stakeholder has been started another specification.
stage, the refinement of physical and interface requirements.
Currently, the approach is oriented to automated consistency
Reaching over the 80% of proxy ports usage for refining
evaluation of requirements specification. However, we plan to
interface requirements of stakeholder and structure elements extend the approach in the near future, to evaluate the
usage for refining physical requirements of stakeholder has been
5
completeness of requirements specification and, finally, we seek Federated Conference on Computer Science and Information Systems,
to combine both to evaluate more precisely the requirements Gdansk, 2016.
specification in model-based systems engineering. [13] T. Avdeenko and N. Pustovalova, "The ontology-based approach to
support the completeness and consistency of the requirements
VI. REFERENCES specification," in International Siberian Conference on Control and
Communications, Omsk, 2015.
[1] S. Friedenthal, A. Moore and R. Steiner, A practical guide to SysML, San
[14] S. K. Kim and D. Carrington, "A formal object-oriented approach to
Francisco: Morgan Kaufmann, 2014.
defining consistency constraints for UML models," in 2004 Australian
[2] S. C. Spangelo, "Applying Model Based Systems Engineering (MBSE) Software Engineering Conference, 2004.
to a Standard CubeSat," in Aerospace Conference IEEE, 2012.
[15] André Pascal, Romanczuk Réquilé Annya, Royer Jean-Claude,
[3] A. Giorgetti, A. Hammad and B. Tatibouët, "Using SysML for Smart "Checking the Consistency of UML Class Diagrams Using Larch Prover,"
Surface Modeling," in First Workshop on Hardware and Software in Rigorous Object-Oriented Methods, York, 2000.
Implementation and Control of Distributed MEMS , France, 2010.
[16] Tsiolakis A, Ehrig H, "Consistency Analysis of UML Class and Sequence
[4] P. M. Institute, "PMI's Pulse of the profession®: Requirements Diagrams using Attributed Graph Grammars," in Proc. of Joint
management-A core competency for project and program success," APPLIGRAPH/GETGRATS Workshop on Graph Transformation, 2000.
Newtown Square, 2014.
[17] Boris Litvak, Shmuel Tyszberowicz, Amiram Yehudai, "Behavioral
[5] Lian Yu, Shuang Su, Shan Luo, Yu Su, "Completeness and Consistency Consistency Validation of UML Diagrams," in First International
Analysis on Requirements of Distributed," in 2nd IFIP/IEEE Conference on Software Engineering and Formal Methods (SEFM'03),
International Symposium on Theoretical Aspects of Software Brisbane, Australia, 2003.
Engineering, 2008.
[18] J. M. Küster, J. Stehr, "Towards Explicit Behavioral Consistency
[6] S. Liu, "Verifying Consistency and Validity of Formal Specifications by Concepts in the UML," in Second International Workshop on Scenarios
Testing," World Congress on Formal Methods in the Development of and State Machines : Models, Algorithms and Tools, Portland, Oregon,
Computing Systems, vol. 1, pp. 712-712, 1999. USA, 2003.
[7] D. Mazeika, A. Morkevicius and A. Aleksandraviciene, "MBSE driven [19] C. Lankeit, V. Just and A. Trächtler, "Consistency analysis for
approach for defining problem domain," in 1th System of Systems requirements, functions, and system elements: Requirements for the
Engineering Conference, Kongsberg, 2016. entire development process," in Annual IEEE Systems Conference, 2016.
[8] A. Morkevicius, A. Aleksandraviciene, D. Mazeika, L. Bisikirskiene, Z. [20] P. Nistala and P. Kumari, "An approach to carry out consistency analysis
Strolia, "MBSE Grid: A Simplified SysML-Based Approach for on requirements: Validating and tracking requirements through a
Modeling Complex Systems," in 27th Annual INCOSE International configuration structure," in 21st IEEE International Requirements
Symposium, 2017. Engineering Conference, Rio de Janeiro, 2013.
[9] R. Cloutier, M. Bone, "Compilation of SysML RFI- Final Report," 20 02 [21] J. Chanda, A. Kanjilal, S. Sengupta and S. Bhattacharya, "Traceability of
2010. [Online]. Available: requirements and consistency verification of UML use case, activity and
https://www.nomagic.com/mbse/images/whitepapers/ Class diagram: A Formal approach," in International Conference on
omg_rfi_final_report_02_20_2010-1.pdf. Methods and Models in Computer Science, Delhi, 2009.
[10] C. Delp, D. Lam, E. Fosse, Cin-Young Lee., "Model based document and [22] A. Goknil, I. Kurtev and K. V. D. Berg, "A Metamodeling Approach for
report generation for systems engineering," in Aerospace Conference, Reasoning about Requirements," Model Driven Architecture –
2013. Foundations and Applications, pp. 310-325, 2008.
[11] P. Godart, J. Gross, R. Mukherjee, W. Ubellacker, "Generating real-time [23] A. Kanjilal, S. Sengupta and S. Bhattacharya, "Analysis of object-
robotics control software from SysML," in IEEE Aerospace Conference, oriented design: A metrics based approach," in IEEE Region 10
2017. Conference, Singapore, 2009.
[12] N. Mahmud, C. Seceleanu and O. Ljungkrantz, "ReSA Tool: Structured
requirements specification and SAT-based consistency-checking," in
6