=Paper=
{{Paper
|id=Vol-1249/paper3
|storemode=property
|title=Model-Driven Software Development of Safety-Critical Avionics Systems: an Experience Report
|pdfUrl=https://ceur-ws.org/Vol-1249/paper3.pdf
|volume=Vol-1249
|dblpUrl=https://dblp.org/rec/conf/models/HovsepyanLbMJRBD14
}}
==Model-Driven Software Development of Safety-Critical Avionics Systems: an Experience Report==
https://ceur-ws.org/Vol-1249/paper3.pdf
Model-Driven Software Development of
Safety-Critical Avionics Systems: an Experience
Report
Aram Hovsepyan1 , Dimitri Van Landuyt1 , Steven Op de beeck1 , Sam
Michiels1 , Wouter Joosen1 , Gustavo Rangel2 , Javier Fernandez Briones2 , Jan
Depauw2 ,
1
iMinds-DistriNet, KULeuven
first.last@cs.kuleuven.be
2
Space Applications Services N.V./S.A.
gustavoenriquerangel@spaceapplications.com, jfb@spaceapplications.com,
jan.depauw@spaceapplications.com
Keywords: dependability and safety, model-driven development process, early
verification and validation
Abstract. The model-driven software development (MDSD) vision has
booked significant advances in the past decades. MDSD is said to be
very promising in tackling the “wicked” problems of software engineering
including development of safety-critical software. However, MDSD tech-
nologies are fragmented as these are typically limited to a single phase in
the software development lifecycle. It seems unclear how to practically
combine the various approaches into an integrated model-driven software
development process.
In this experience report, we present an end-to-end MDSD process that
supports safety-critical software development from the point of view of
Space Applications Services, an industrial aerospace company. The pro-
posed development process is a bottom-up solution based on the state
of the practice and the needs of Space Applications Services. The pro-
cess integrates every software development activity starting from require-
ments definition all the way to the verification and validation activities.
Furthermore, we have created an integrated toolchain that supports the
presented MDSD process. We have performed an initial evaluation of
both the process and the toolset on a case study of an On-Board Control
Procedure Engine.
1 Introduction
Given the advances in the hardware technologies software development in gen-
eral is becoming an increasingly complex activity. Building software for avionics
systems is posing an even bigger challenge as dependability and safety are con-
cerns of paramount importance. Dependability refers to how software failures
can result in a degradation of the system performance or even in loss of mis-
sion or material. Safety, on the other hand, is defined as a system property that
is concerned with failures that can result in hazards to people or systems. For
safety-critical systems it is often compulsory to perform various safety-related
analyses as part of the software development lifecycle.
The model-driven software development (MDSD) vision seems very promis-
ing in efficiently tackling the essential complexities (including safety concerns)
of the software development process [1]. The MDSD vision, primarily focused on
the vertical separation of concerns, aims at reducing the gap between problem
and software implementation domains through the use of models that describe
complex systems at different abstraction levels and from a variety of perspec-
tives. Various standards, tools and techniques that are well-aligned with the
MDSD vision are currently becoming widely accepted by the industry. The Ar-
chitecture Analysis & Design Language (AADL) is a de-facto standard in the
domain of avionics and automotive software systems. The use of AADL enables
various types of analyses that link to dependability and safety aspects (e.g.,
schedulability analysis). SysML is a standard general-purpose language for sys-
tems engineering. SysML could be used to plug-in certain safety-related analyses,
such as Software Failure Mode, Effects & Criticality Analysis (SFMECA) and
Software Fault Tree Analysis (SFTA) [2]. While these techniques contribute to
the aspect of safety, they are all focused on a specific phase of the software
development lifecycle. As a consequence, these tools and approaches are frag-
mented and it remains unclear how these approaches can be chained together to
form a complete MDSD development process and toolchain. There is a lack of a
pragmatic model-based software development process that provides a complete
software lifecycle and transparently integrates the various building blocks. The
required process should enable model-based software development starting from
requirements analysis all the way to the verification and validation activities
of the final implementation. Finally, the transitions and traceability links be-
tween the different phases in the development lifecycle should be automatically
managed.
In this paper, we present our experiences with designing a complete MDSD
process in collaboration with Space Applications Services (an independent Bel-
gian space technology company). Our contributions are twofold. Firstly, we have
created an end-to-end MDSD process that covers all phases of software de-
velopment lifecycle and focuses explicitly on safety and dependability aspects.
The proposed end-to-end process is conform with a set of guidelines for embed-
ded and real-time software development prescribed by European Space Agency
(ESA) [3]. The end-to-end development process leverages the V-model and the
DSDM Atern agile framework [4]. We use design models for incremental skele-
ton code generation. Moreover, the proposed integrated process provides the
necessary mechanisms to perform several critical architectural analyses, i.e.,
SFMECA/SFTA and various analyses enabled by the use of AADL. Secondly, we
have successfully integrated a number of tools that enable the proposed MDSD
process. The presented approach is currently being validated by Space Applica-
tions Services in the development of a spacecraft On-Board Control Procedure
Engine (OBCP) [5].
The remainder of the paper is structured as follows. In section 2, we provide
background information on relevant dependability- and safety-related standards
and techniques. We also describe the problem statement in detail. In section
3, we present our solution in detail and discuss its advantages and drawbacks.
Section 4 presents a number of related works. Finally, section 5 concludes this
paper.
2 Background
To develop software in the avionics domain, software engineers must not only
develop complex real-time software, but also place the safety and dependability
qualities in the driver seat. Furthermore, certification plays an essential role in
avionics software otherwise not present in many other domains. The dependabil-
ity, safety and certification concerns pose a significant challenge as they affect
each phase of the software development lifecycle. In this section, we provide an
overview of these concepts. Then we briefly outline a number of methodologies
and techniques that focus on specific aspects related to dependability and safety.
In addition, we summarise how these activities are typically performed within
Space Applications Services. Finally, we present the problem statement in detail.
2.1 Dependability and Safety
Dependability and safety are key concerns in the development and operations
of avionics systems. The contribution of software to system dependability and
safety is a key factor given the growing complexity and applicability of software in
avionics applications. Dependability is concerned with software reliability, avail-
ability and maintainability. Software reliability is the property of software of
being “free from faults” [6]. A fault can be a result of human mistake made in
requirements specification, design specification, coding or even, mistakes made
while executing the software development process. In general, faults can lead
to errors that can lead to failures, i.e., an unexpected/unintended behaviour of
the system. Software maintainability relates to the ease with which the software
can be modified and put back into operation. Finally, software availability is the
capability of the software to perform a required function at a given instant of
time (or time interval). Safety is concerned with those failures that can result
in actual system hazards (as opposed to software reliability that is concerned
with all software failures). Safety is a system property. Nonetheless, software is a
main component of a system, and therefore contributes to its safety. As opposed
to typical software development, avionics software must undergo a certification
process before its utilisation. Safety certification assures that deployment of a
given system does not pose an unacceptable risk of harm. Furthermore, safety
certification is also concerned with the quality of the development process and
all its intermediary artifacts, such as requirements, architecture, etc.
2.2 Safety Analysis Activities
A number of tools and techniques exist that focus on dependability and safety.
These techniques are typically applied in very different stages of the software
development process. This section briefly describes three methodologies that are
highly relevant within the domain of applications developed by Space Applica-
tions Services. It is not our intention to be exhaustive in listing the relevant
approaches.
Software Failure Modes, Effects and Criticality Analysis (SFMECA)
is an iterative activity, intended to analyse the effects and criticality of failure
modes of the software within a system [7]. The analysis provides an essential
contribution to the development of the product architecture and to the definition
of the test and operation procedure. The result of the SFMECA analysis is a
table that contains the failure, function, failure mode, effect, criticality, impact,
action and mitigation. This analysis method can reveal failures not detected
by system level analysis. Furthermore, SFMECA analysis can identify critical
components, support design and verification decisions. It is essential that such
decisions are easily traced back from the latter software development phases to
their original artifacts.
Software Fault Tree Analysis (SFTA) is a deductive, top down method for
analysing system design and performance [7]. It involves specifying a top event
(also referred to as ”feared event”) to analyse, followed by identifying all of the
associated elements in the system that could cause that top event to occur.
SFTA is a logical and structured process that helps identify potential causes of
system failure before the failure actually occurs. The resulting output of SFTA
is a fault tree, describing the potential faults in the software.
2.3 Problem Statement
From the Space Applications Services point of view the current state of practice
in MDSD suffers from three drawbacks that play a significant role in MDSD
adoption.
Lack of an Integrated Process/Toolchain. Despite the clear advances in
the state-of-the-art, MDSD research methodologies and techniques typically stay
focused on a specific phase in the software development lifecycle. It remains quite
unclear how to produce a software system starting from customer requirements
all the way to a validated and verified implementation. While a one-size-fits-all
approach is unlikely to provide a systematic solution, we believe that a collection
of pragmatic bottom-up solutions is essential for the mainstream adoption of
MDSD. This problem relates to both an end-to-end process as well as a toolchain
that supports this process in a MDSD context.
Lack of Safety Engineering Methodology Integration. Even if a UML-
centric end-to-end process seems feasible given the MDSD tools, avionics soft-
ware systems must adhere to stringent safety standards. In the previous section,
we have briefly described a number of safety analyses and methodologies that
tackle a specific dependability and/or safety related aspect of the system. Space
Applications Services currently performs both SFMECA/SFTA analyses manu-
ally by leveraging Office-like (e.g., Powerpoint/Excel) applications. Ideally, ar-
chitecture and design models (with some additional annotation for feared events
and causing/contributing factors) could be used to run SFMECA/SFTA analy-
ses. Real-time performance analyses, such as schedulability analysis, end-to-end
flow latency analysis, are automated, but performed only at the implementation
level. The use of architecture-level analyses enabled by AADL could allow Space
Applications Services to early verify and validate all design decisions. The inte-
gration of all these activities in a hypothesised UML-centric end-to-end MDSD
process is not obvious.
Lack of End-to-End Process Traceability. Traceability plays an essential
role in the domain of avionics systems especially in the context of the certifi-
cation process. Indeed, it is crucial to have the necessary abstractions to trace
each code-level entity back to a set of requirements. An end-to-end MDSD pro-
cess introduces additional challenges as traces should ideally provide complete
information regarding the code, model and requirements interrelations.
3 Space Applications Services Development Process
In this section, we present a prototype end-to-end development process that
provides a pragmatic answer to the challenges outlined in the previous section.
The proposed approach is inspired by the engineering process proposed by the
European Space Agency (ESA) for the development of embedded and real-time
on-board software. We also briefly mention a number of tools that provide the
backbone of the proposed process.
3.1 Software Development Process
ESA has introduced a standard engineering process relevant to all space ele-
ments of a space system [3]. The phases covered by the standard are as follows.
Requirements Baseline corresponds to the complete specification provided
by the end-user regarding the software product expectations. Technical Re-
quirements correspond to all technical aspects that the software shall fulfil
with respect to the end-user requirements. Software Architecture Design
corresponds to the overall architecture that is created and refined based on
the technical requirements. Software Component Design corresponds to a
more detailed description of the elements described by the software architec-
ture. Implementation corresponds to the development of the various software
components described in the software component design phase. Verification
corresponds to the testing of produced implementation in order to verify the
correctness of the product performance. Validation corresponds to the testing
of the software components as well as the complete software in order to validate
the correctness of product performance.
We have created an integrated development process that is based on the no-
tion of the V-Model and the Agile Dynamic System Development Method Atern
framework. Figure 1 illustrates a structural view of the development process
that presents each development activity along with their structural connections
to other activities. This process is in line with the V-Model. Our contribution is
represented in grey by the two additional activities, i.e., SFMECA/SFTA and
AADL analyses, that cut across multiple development phases. The lines between
each activity schematically represent not only the process flow, but also the
artifact exchange between activities. For instance, technical requirements anal-
ysis is preceded by the software architecture design. Ideally, each requirement
is known and accessible at the architectural level. This enables the creation of
traces (or the traceability information) that link architectural elements to their
corresponding requirements. The traceability information between different de-
velopment phases is essential as it enables early requirements validation. Note
that the process is inherently iterative and one can always go back to an earlier
activity. This information was dropped from figure 1 for readability purposes.
Requirements Validation
Baseline (Acceptance)
Rational DOORS Rational DOORS
Technical Verification
Requirements (Test Cases)
Rational DOORS Rational DOORS
SFMECA/SFTA
Safety Architect
Software Architecture Verification
Design (Integration Tests)
MagicDraw Architectural VectorCAST
Dependability Analysis
AADL - OSATE
Software Component Verification
Design (Unit Tests)
MagicDraw VectorCAST
Traceability
information Implementation
Eclipse (C++
Artefact or process editor, compiler)
flow
Fig. 1. Space Apps V-Model Software Development Process
For the dynamics of this process we leverage the Dynamic System Develop-
ment Method (DSDM) Atern agile project delivery framework used for software
development. The idea behind DSDM is to develop a solution iteratively starting
from global view of the product. For a detailed description of DSDM Atern, we
refer the interested readers to [4].
3.2 Integrated Toolchain
We further briefly outline the tools currently utilised within Space Applications
Services that support the presented structural process. We also provide infor-
mation how software development artifacts are interchanged between the tools.
Figure 2 presents the tools for each activity. Note that some Space Applications
Services’ customers require the use of Rational DOORS during the software
development. However, all other tools can be freely replaced by alternatives.
Requirements Baseline
Technical Requirements
Validation
Rational DOORS
Software Architecture Design SFMECA/SFTA Verification
Software Component Design Safety Architect (Unit Tests, Integration Tests)
MagicDraw/Cameo DataHub VectorCAST
traces Architectural Dependability Analysis
Code Generation
Artefact flow Implementation
MERgE Platform
(OSATE, MOFScript, CDT)
Fig. 2. Structural Software Development Process Toolchain
IBM Rational DOORS tool is used for the the Requirement Baseline,
Technical Requirements and Validation Activities.
MagicDraw tool is used for the Software Architecture Design and Software
Component Design phases. The data interchange between Rational DOORS and
MagicDraw is realised by the Cameo DataHub tool that features a complete syn-
chronisation of requirements as well as traceability links between the tools. Note
that MagicDraw features a built-in functionality to both export and import all
modelling artifacts to the Eclipse Modelling Framework (EMF). EMF [8] pro-
vides the common infrastructure and a de facto UML standard implementation
for the model interchange between various tools.
Safety Architect is used for the SFMECA/SFTA Analyses activities [2]. Eclipse
EMF is used as a common language to interchange models.
MERgE Platform is an Eclipse-based toolset that provides an integrated col-
lection of plug-ins. We leverage the MOFscript [9] plug-in for transforming
the UML models into their AADL representation. We use the UML MARTE
profile for annotating the UML model elements with real-time and embedded
properties [10]. The AADL model is used by the OSATE tool for performing
Architectural Dependability Analysis. We also use MOFScript to generate skele-
ton C code that is further incrementally refined into a working implementation.
Eclipse CDT plug-in provides the necessary tools for the C code implementa-
tion. VectorCAST is used for the three Verification phases it automates unit
and integration testing activities.
3.3 Evaluation
In collaboration with Space Applications Services, we have performed an ini-
tial evaluation of the proposed process and toolchain on the case study of an
On-Board Control Procedure Engine. The MDSD process is considered to be
complete in the sense that it covers all phases from a software development life-
cycle required from the Space Applications Services point of view. The software
artefacts integration throughout the various phases is either provided by the tool
(e.g., Requirements and Technical requirements in Rational DOORS, or Software
Architecture and Software component Design in MagicDraw) or automatically
transformed by additional tools (e.g., MOFScript). Moreover, all the transitions
support the incremental nature of the complete process. This is essential as exist-
ing artefacts shall not break by subsequent iterations (e.g., manual refinements
to the AADL models or the generated code must be preserved). We have success-
fully integrated a number of AADL analyses by implementing ideas presented
in [10]. The integration of safety-related analyses (i.e., SFMECA and SFTA) are
currently work in progress. Finally, we have provided an initial implementation
towards tackling the traceability challenge. The traceability of various elements
between Rational DOORS and MagicDraw are actually provided by the tools.
The traceability between MagicDraw and the MERgE Platform is implemented
by incorporating references to the MagicDraw modelling elements as comments
both in the generated AADL model as well as in the generated code.
While the proposed approach seems pragmatic and effective in tackling the
challenges described in section 2.3 we still face a number of challenges that are
work in progress. At the toolchain level, we are still working towards integrating
the SFMECA/SFTA analyses in the Safety Architect tool. At the process level,
we are facing the challenge of having a process that contains many implicit con-
straints. If these constraints are not correctly followed the process may become
completely useless. For instance, source code should not be manually refined
without encoding that information into the detailed design as subsequent iter-
ations may break the manual code. This problem can be efficiently tackled by
using a Process Modelling Language (PML), such as OMG SPEM [11]. However,
even if such constraints are made explicit it is impossible to capture all possi-
ble situations. Furthermore, developers always deviate from the process model
either because of lack of experience or the imperfections of the proposed pro-
cess. Da Silva et al [12] present a systematic approach agnostic to a particular
PML selection to deal with such deviations. Our end-to-end MDSD approach
could substantially benefit from an integration with such a systematic frame-
work. Finally, a challenge both at the process and at the toolchain level remains
the management of the traceability information between various entities across
different abstraction levels, which is currently somewhat implicit. Ideally, a sys-
tematic approach should allow the transparent management of all traces within
a separate view.
4 Related Work
A number of research efforts focused on architecture optimisation are comple-
mentary to our work as they would enhance the Software Architecture Design
and Architectural Dependability Analysis phases. Etemaadi et al have presented
an approach with a supporting toolkit named AQOSA to support architecture
optimisation with respect to quality attributes [13]. Meedeniya et al have ad-
dressed the problem of evaluating reliability based on software architectures in
the presence of uncertainty [14]. Brosch et al have introduced a reliability mod-
eling and prediction technique that considers the relevant architectural factors
of software systems by explicitly modeling the system usage profile and execu-
tion environment and automatically deriving component usage profiles [15]. As
opposed to these research initiatives our approach is focused more on the overall
development process rather than a specific development phase.
Several research initiatives are focused on providing a systematic methodology
for tool integration. Balogh et al have proposed a workflow-driven tool integra-
tion framework using model transformations that allows one to formally specify
contracts for each transition between tools in the tool chain [16]. Klar et al have
created a meta-model-driven environment that allows to integrate tools by focus-
ing on traceability links between dependent tool artifacts [17]. The approach we
have proposed in this work could be seen as a case study for the tool integration
frameworks.
5 Conclusion
Developing software for the avionics domain is an extremely challenging task
given the strict dependability and safety requirements. The advent of Model-
Driven Software Development (MDSD) standards and tools has substantially
improved the current state-of-the-art by introducing a number of systematic
disciplines throughout various stages of the software development lifecycle. Un-
fortunately, these techniques are currently fragmented and it remains unclear
how these could be combined into an integrated end-to-end software develop-
ment process. In this experience paper, we have proposed a complete MDSD
process inspired by the V-model that integrates the various standards and tools
into a single integrated software development process. The proposed process
is based on the needs and experiences within Space Applications Services, an
industrial aerospace company. The MDSD process integrates transparently a
number of standards from the aeronautics domain, such as architectural analy-
sis and design language (AADL), software failure modes, effects and criticality
analysis (SFMECA) and software fault tree analysis (SFTA). Finally, the end-
to-end MDSD provides an initial answer to the traceability requirements within
Space Applications Services. In the future we plan to integrate the MDSD pro-
cess with a systematic process modelling and deviation detection and resolution
framework. We are also looking to improve the integration of traceability infor-
mation. Finally, we plan to further validate the proposed process on the case
study of an On-Board Control Procedure Engine.
Acknowledgements
The presented research is partially funded by the Research Fund KU Leuven and
the Flemish agency for Innovation by Science and Technology (IWT 120085).
The research activities were conducted in the context of ITEA2-MERgE (Multi-
Concerns Interactions System Engineering, ITEA2 11011), a European collabo-
rative project with a focus on safety and security [18].
References
1. France, R., Rumpe, B.: Model-driven development of complex software: A re-
search roadmap. In: Proceedings of the 29th International Conference on Software
Engineering, IEEE Computer Society (2007) 37–54
2. All4Tec: Safety architect. (http://all4tec.net/index.php/en/model-based-safety-
analysis)
3. ECSS Space Engineering: Safety. ECSS-E-ST-40C. Misc (2009)
4. Consortium, D.: The DSDM Atern Handbook. DSDM Consortium (2008)
5. ECSS Space Engineering: Spacecraft on-board control procedures. ECSS-E-ST-70-
01C. Misc (2010)
6. ECSS Space Engineering: Software dependability and safety. ECSS-Q-HB-80-03A.
Misc (2012)
7. Jet Propulsion Laboratory: Software Fault Analysis Handbook. (2005)
8. Eclipse: Eclipse modeling framework (EMF). (http://www.eclipse.org/emf/)
9. SINTEF: MOFScript. (http://modelbased.net/mofscript/)
10. Faugére, M., Bourbeau, T., de Simone, R., Gérard, S.: MARTE: Also an uml profile
for modeling AADL applications. In: ICECCS, IEEE Computer Society (2007)
11. OMG: Software Process Engineering Metamodell SPEM 2.0. Technical report,
OMG (2006)
12. da Silva, M.A.A., Bendraou, R., Blanc, X., Gervais, M.P.: Early deviation detection
in modeling activities of mde processes. In: MoDELS. (2010) 303–317
13. Etemaadi, R., Lind, K., Heldal, R., Chaudron, M.R.V.: Quality-driven optimiza-
tion of system architecture: Industrial case study on an automotive sub-system.
Journal of Systems and Software (2013) 2559–2573
14. Meedeniya, I., Moser, I., Aleti, A., Grunske, L.: Architecture-based reliability eval-
uation under uncertainty. In: Proceedings of the Joint ACM SIGSOFT Conference.
QoSA-ISARCS ’11, New York, NY, USA, ACM (2011) 85–94
15. Brosch, F., Koziolek, H., Buhnova, B., Reussner, R.: Architecture-based relia-
bility prediction with the palladio component model. Transactions on Software
Engineering (2011)
16. Balogh, A., Bergmann, G., Csertán, G., Gönczy, L., Horváth, Á., Majzik, I., Patar-
icza, A., Polgár, B., Ráth, I., Varró, D., Varró, G.: Workflow-driven tool integration
using model transformations. In: Graph Transformations and Model-Driven Engi-
neering. Lecture Notes in Computer Science, Springer (2010) 224–248
17. Klar, F., Rose, S., Schürr, A.: TiE - a tool integration environment. In: Proceedings
of the 5th ECMDA Traceability Workshop. Volume WP09-09 of CTIT Workshop
Proceedings. (2009) 39–48
18. MERgE Consortium: MERgE: Multi-concerns interactions system engineering.
(http://www.merge-project.eu)