=Paper=
{{Paper
|id=Vol-2581/aviose2020paper1
|storemode=property
|title=Approach to Systematic Test Signal Definition forOperation Scenarios of Aircraft Systems
|pdfUrl=https://ceur-ws.org/Vol-2581/aviose2020paper1.pdf
|volume=Vol-2581
|authors=Dennis Hillig,Frank Thielecke
|dblpUrl=https://dblp.org/rec/conf/se/HilligT20
}}
==Approach to Systematic Test Signal Definition forOperation Scenarios of Aircraft Systems==
https://ceur-ws.org/Vol-2581/aviose2020paper1.pdf
Approach to Systematic Test Signal Definition for
Operation Scenarios of Aircraft Systems
1st Dennis Hillig 2nd Frank Thielecke
Institute of Aircraft Systems Engineering Institute of Aircraft Systems Engineering
Hamburg University of Technology (TUHH) Hamburg University of Technology (TUHH)
Hamburg, Germany Hamburg, Germany
Dennis.Hillig@tuhh.de Frank.Thielecke@tuhh.de
Abstract—With rapidly increasing levels of automation, mod- identified, when the system complexity rises. Firstly, it be-
ern aircraft systems become increasingly complex. The designed comes difficult to find the relevant scenarios or operational
solutions typically achieve multiple functions by using a highly conditions. Secondly, the implementation of the realistic test
integrated mix of hardware and software components. With
regard to automation, particularly the number of software vectors complicates significantly.
components increases. In order to approach this challenge for aircraft systems, a me-
For verification and validation activities, such as testing, this thodical procedure for a structured and model-based scenario
intensifies the challenge to define all relevant scenarios to test a definition was developed at the Institute of Aircraft Systems
system’s functionalities and particularly its robustness in the case Engineering of the Hamburg University of Technology. It
of unusual, but realistic situations. Hidden failure cases might be
missed. Additionally, it becomes infeasible to systematically define is intended to cope with the system’s complexity through
the test vectors to study a systems behavior thoroughly due to the the main ideas of modularisation and hierarchical refinement.
immense test space. As a consequence, testing gets more work- The contribution is embedded into the activities to develop a
and time-intensive. seamless avionics tool chain (s. [1]).
This paper presents a stepwise model-based method to define and The aim of the developed method is clarified with an example
refine the stimulating part of the test scenarios in a systematic and
modular manner to approach this challenge. The aim is to work use case of an air conditioning system in the following. Figure
out diverse and relevant operational scenarios to validate system 1 shows an overview of the system under test (SUT) and
functions and to set up realistic test vectors for these scenarios. some interfacing elements, such as the aircraft’s environment
The paper further focuses on the aim to facilitate automation and conditions, cabin loads and cockpit commands from the over-
re-usability of scenario and functional signal elements to enable head panel. The system is modeled in an open-loop manner
continuous testing at different stages of the development process.
Concepts of a developed prototype are outlined as well. Finally, w.r.t. the interfacing elements. The actions of these interfacing
the current state of the method is discussed and an overview of elements should be explicitly integrated in the scenario def-
the way forward is given. inition. Given that the physical system was designed with a
Index Terms—scenario testing, aircraft systems, test design representation as a physical model, and the control and monitor
(C&M) functions (e.g. pressure control) were designed and
I. I NTRODUCTION implemented, the scenario-based test has the following two
In recent years new digital and smart embedded system objectives: On one hand, the system hardware design should be
solutions emerged at a very rapid pace. Such solutions are verified, e.g. in terms of sizing and functionality. On the other
increasingly used to optimize novel aerospace systems, which hand, the software functions should be validated under realistic
integrate mechanical, mechatronical and software components, operational scenario conditions. To achieve that, realistic input
through automation. However, the more capabilities and func- signals or models of the interfacing elements are needed for a
tions these integrated intelligent solutions implement, the more given set of scenarios, which also have to be identified. This
apparent the need becomes for new verification and validation task is not trivial, particularly as interfaces include numerous
(V&V) methods, that can handle the complexity. In particular continuous signals.
for safety-critical systems this is crucial. In that context, The presented definition process should further facilitate au-
classical approaches for the examination of system’s integrated tomation and reuse at different stages of the development.
functions and safety, which rely significantly on manual testing With regard to the latter, system component models could
and expertise, are a limiting factor to new developments. be replaced by real hardware, for example, and the scenarios
Testing in general is the mainly used V&V method. Scenario- should still be applicable with minimal necessary adaptions.
based testing more specifically is one technique to assess A prototype editor for the presented mission scenario and
the operational, functional behavior at system and overall stimulating test signal definition was developed using Mat-
aircraft level. Scenario-like analyses of technical systems are lab/Simulink. Applied concepts are integrated in the following
also commonly used for demonstrations of functionality in explanations. Specified information can also be captured as an
an operational context. Yet, two main challenges can be .xml exchange document.
Copyright © 2020 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
tries. Therefore, these scenarios are often further classified e.g.
as positive or negative w.r.t. whether the goal of the interaction
is expected to be achieved or not. A trivial aircraft example:
If the aircraft is on ground, it should not be possible to retract
the landing gear.
As outlined here, scenario testing has the potential to assure
safer and more robust system-functions through the diversity
of realistic scenarios. Particularly automation functions could
be validated early. However, it is also clear that a systematic
test of the complete input-output space is not the goal. A
scenario targets one specific, but complex set of operations
and exploring all possible parameter variations would blow
up the number of test cases. Other approaches to black-box
testing probably perform better in this respect. Moreover, as
Fig. 1. Example usecase: Air conditioning system. a black-box testing technique, coverage of all internal system
states is also not the primary intend where a structure-based
technique may be more suitable.
The paper firstly presents the fundamentals of scenario-based
testing and the applied mission-based scenario approach. B. Mission-Based Scenario Approach
Subsequently the developed scenario definition process is
As aircraft systems are typically integrated subsystems of
outlined. This includes an initial methodical overview as well
the overall vehicle, the functional behavior is typically oriented
as the different fundamental steps being scenario specification,
towards the flight mission. For that reason, an operational
system and environment definition and signal and component
scenario-based test approach was developed in [5], which
implementation. Finally, a short conclusion is given and the
models the flight mission in a state-chart as a sequence of
future work is described.
fundamental phases, that are represented as atomic states
II. F UNDAMENTALS along a scenario path. The major aim is to support the
A. Scenario Testing overall system integration tests to observe coupling effects
between involved components and software functions. The
The classical scenario test approach can be categorized current application focuses on the use for simulation-based
as a specification-based test design technique (s. [2]). As pre-design investigations but the principal transferability to
equivalently called ”black-box testing” this means that require- final integration testing is desired. Therefore, the approach
ments, specifications or (interface) models etc. are used to intends to complement requirements-based test activities by
derive test cases without knowledge of the internal structure exploring the operational overall system behavior to uncover
of the tested system [2]. In that respect scenario testing undesired effects, with a particular desire to find interactions
focuses on sequences of interactions with interfacing systems between ATA chapters.
or components. In software testing literature, also user-centric The virtual test architecture from [5] foresees the partition of
use cases are utilized equivalently to scenarios [3]. Other stimulating ’test cases’ though the scenario state-chart and the
specification-based testing methods contrarily mainly rely on evaluation, which is done by observing agent modules running
the systems input-output space for specification verification in parallel. These modules could check arbitrary functional
e.g. by identifying possible input combinations or partitions behavior or performance of the system. With this approach
with equivalent behavior. Therefore, scenario testing is more it is possible to realistically set up operational tests, which
suitable to assess systems long-running (end-to-end) behavior enable assessment the overall system’s functional behavior
in a realistic operational situation. This story-like character during complete flight missions. As the integrated system
consequently attains motivational and meaningful context (s. is investigated, multiple functional specification aspects and
[4]) to discover hidden failures and their possible effects. operational sequences can be observed in an simultaneous
Models, e.g. formulated in formalisms like state charts or manner.
sequence diagrams, are commonly used to describe the pos- The scenario concept presented in [5] and [6] uses multi-
sible interaction sequences. The typical scenario definition signal segments in time-series format from a segment library to
procedure from [2], towards which this work is oriented, can create continuous signal profiles for each phase. Test signals
be reduced to the following two steps: set up for each phase separately are thereby composed to a
1) Definition of one main scenario, that describes the complete test vector representing the flight mission scenario.
nominal sequence of actions To cover multiple scenarios, the concept allows branching of
2) Derivation of alternative scenarios based on the given the scenario paths. The time-continuous signal profiles defined
main scenario. within the phase are complemented by parallel events that are
The latter can be diverse, as it includes foreseen operational triggered at discrete test points during a phase. The events
options, exceptions and intended or unintended incorrect en- could for example represent cockpit commands and can trigger
alternative phase transitions of the scenario path. This enables (and timeouts) may be used to model environment reactions
the modeling of events such as an emergency during cruise and on system outputs. The presented approach is understood to
resulting alternative flight phases, such as emergency landing be more explorative and investigative in the sense to move
and evacuation. away from the scenarios that are explicitly defined through the
In [5] and [6] the outlined concept was successfully demon- specification. It should be emphasized that running scenario-
strated at a multi-functional fuel-cell system, which included based specification models in parallel to observe required
control and automation functions. However, the complex sce- functional behavior is possible and intended but exceeds the
nario state-chart has to be created manually and is hardly scope of this paper.
reusable. The presented method in this paper also applies the 2) Automotive Street Scenarios: In the context of the vali-
same mission-based scenario approach. With the foundations dation of highly automated driving functions in the automotive
from [5], the method tries to offer a more structured, modular sector, the works in the context of the PEGASUS research
and reusable process for the definition of the stimulating project [15] follow a similar operational scenario philosophy
scenario state-chart, that can be automated further. as presented in this paper. Street scenarios, defined in a stan-
dardized and structured format [14], include static content of
C. Related Work the street situation and dynamic elements such as moving cars
1) Testing using Scenario-Based System Specifications: or pedestrians. The concept allows to execute the scenarios
Several works in the recent past adressed the model-based at simulation as well as real world level. Scenarios can also
specification of system behavior in the form of scenarios and be to remodeled from real operation for further investigation.
the test activities on the basis of this specification. To clarify Nevertheless, a difference for the application in the aircraft
the distinction in the light of scenario-based testing, some domain is the diversity and number of systems in comparison
works in that field are discussed in the following. to the variety of traffic situation in the automotive sector.
For the specification of reactive system behavior, message 3) Signal Segments in Testing: In [16] a segment-based
sequence diagrams are often used in literature on software approach is presented to generate complex signal inputs with
engineering to visually define wanted and unwanted system a low number of parameters for each segment. This principle
scenarios. This particularly applies in the context of testing. is similar to the definition of signal functions in the context of
One example is the standardized test specification language this work. However, the work in [16] conducts a structural test
TTCN-3 [17] with primary applications for communication of automotive Simulink models using a search-based algorithm
systems, which allows the formulation of test cases in the and does not predefine operational scenarios.
form of sequence diagrams. It is pointed out in [9], that
III. S CENARIO D EFINITION P ROCESS
the formalism has the advantage to focus on inter-object
communication scenarios without having to specify the objects The developed definition process pursues three basic incen-
internal behaviors. Moreover, extensions such as live sequence tives following the modularization and hierarchy principles:
charts (LSC) [12] exist, which allow the precise definition 1) Easy set up and management of complex scenario tests
of system specifications with different modalities, e.g. what makes them less error prone and more complete for
must and what may happen. Such sequence charts are also increasingly complex systems.
extended in research to describe time constraints (s. [11]) or 2) Give context and improve automated analysis and doc-
generally visualize temporal properties such as linear temporal umentation by association of signal segments with spe-
logic (LTL) formulas (s. [13]). cific scenario story elements or actions of the SUT and
The work in [10] further demonstrates a way how automatic its interfacing environment elements.
test generation can be achieved with modal scenario speci- 3) Maximize the reuse of scenario tests and its elements to
fications. The approach allows the specification of multiple reduce effort and make tests more comparable.
sequence charts for the system functions that can be active The concrete concepts are discussed in the following.
in parallel, if an initial sequence of the respective scenario
occurs. Due to the granularity of scenario definitions, the A. Methodical Process Overview
method seems quite scalable. Moreover, a coverage criterium The developed procedure is structured into 5 fundamental
was developed to minimize redundant scenario combinations steps. An overview of these methodical steps for the scenario
and thus the amount of test cases. definition is depicted in Fig. 2. The steps can be grouped into
However, the scenario approach of this work is principally three types of activities: Scenario specification, system and
different. As mentioned in the previous section it should environment definition as well as signal assignment.
complement the test activities on the basis of specified require- The first abstracts from the concrete system interface or signals
ments. Instead of using a specification model of the system and tries to identify all relevant mission scenarios and their
the presented approach models the stimulating environment logical sequence of phases and events. At the end of this
in an operational context on aircraft level. Expectations on activity, the structural definition of the state chart is complete
system’s behavior are not modeled in the current scenario and specified up to its basic phase elements and a graphical
state-chart and must be evaluated in parallel observation representation can be produced (e.g. in Matlab/Simulink State-
modules or manually in post processing. Reactive behavior flow). Furthermore, different scenario test cases can now be
Fig. 2. Methodical steps in the scenario definition process.
derived on the basis of the transition paths through the state- cess is completed. Enough information is gathered to create
chart. However, as the basic phase states are still empty and executable scenario test cases. Due to the generic setup, an
no interface signals are defined yet, the test cases cannot be export to arbitrary test languages is principally possible. The
applied. implemented prototype realizes executable test cases as state
The second activity defines the concrete SUT interface and its charts in Matlab/Simulink Stateflow and the use with Simulink
environment elements. When finished, all interface signals for Test is foreseen. Furthermore the export to the state machine
the system under test are registered and grouped according notation SCXML [7] was tested, which is currently further
to potential environmental elements. These elements can be extended as a generic exchangeable test language in a research
represented as parallel states within a compound phase state. project.
The scenario test cases defined at this stage could be executed, A more detailed description for the concepts in each step is
but do not exercise the scenarios correctly, as no signal profiles presented in the following.
are defined yet.
B. Scenario Specification Layer
The third and final activity assigns concrete signal profiles
to the defined signals. For each phase and each environment The scenario specification layer has the intention to identify
element, sequences of signal segment models can be assigned. and structure all relevant operational scenario conditions by
For re-usability, these models could be linked from a library as performing the two steps Scenario registration and Definition
parameterized elements. In such way generic signal segments, of the scenario statechart.
such as a ramp or set functions can be applied, as well as very 1) Scenario Registration (AC/System-level):
specific profiles such as the pressure sensor signal including In this initial step, the scenarios are registered by a tex-
noise during an depressurization event. tual definition and classified. The primary classification is to
Following the signal assignment the scenario definition pro- whether a scenario represents a nominal sequence of events
or an alternative one. Further classification of the latter e.g. as
positive, exception, negative and misuse with context to system
and aircraft functions still need to be worked out. Optionally
a classification for expected frequency of occurrence or links
to requirement elements can be inserted. Such additional
information should later be used for test case prioritisation
and filtering.
For the mission-based scenario approach, two different cate-
gories of scenarios were identified:
• Aircraft mission scenarios
• System operational scenarios
The first includes scenarios such as nominal and alternative
flight phases (e.g. loiter in approach) or emergency descent.
The latter focuses on the system functions and exploits differ-
ent system operations or configurations and internal or external
failures.
2) Stepwise Definition of the Scenario State-Chart:
As a second step, each scenario is setup in one integrated
state chart structure. Scenarios are implemented by sequences
of phase states or event triggers, that could occur during
selected phases or phase groups (e.g. in flight) at defined test
points, or combinations. Firstly one flight-mission phase path
is defined for the main nominal scenario. This scenario should
equivalently represent the nominal system operation. Subse-
quently the alternative scenarios can be defined by branching
the nominal scenario at a selected phase and proceeding with
alternative mission phases and by defining parallel trigger
events that mark the start of the scenario. Phase end conditions
could be defined textually for the transition between the
phase states, which then have to be refined by state chart
variables later. Scenarios can partially share a scenario phase
path and end by final phases or explicitly branching into
Fig. 3. Exemplary scenario state chart (in orientation to [5]).
other scenario paths (e.g. alternative flight mission with the
same landing procedure). An implemented example for the
developed prototype is depicted in Figure 3 - 5. As shown in
They occur at discrete test points during the phases. In the
Figure 4, a scenario path, such as the nominal flight mission
current concept and in orientation to [5], test points apply
in the given example, can be constructed with regard to the
for phases that have a fixed minimum execution time defined
taken transition path. Alternatively, as depicted in 5 for the
(e.g. cruise time). By defining an integer sampling number for
emergency case, a tabular view of the state path could be
each phase, the given time frame is partitioned in as many
used. The second figure shows, how the alternative scenario
parts, where each partitions beginning marks a test point. The
is constructed by branching out of the nominal one. This
defined sampling number therefore has a significant effect on
structure-based editing can be further complemented by a
the resulting count of test cases.
hierarchical tree view for all existing states, given on the
Several parallel events could occur in sequence for one sce-
left side of the figures. In addition to this structural edition
nario when suitable transition conditions are defined (e.g.
of scenarios, the illustrated conceptual prototype realizes a
failure followed by a corrective cockpit action after a defined
graphical state machine view in Matlab/Simulink Stateflow.
time). However, it needs to be taken care that transitions to
As both perspectives are transferable, a graphical state chart as
other phases in between these events do not happen.
depicted in Figure 3 can be set up easily by this hybrid editing
A method still needs to be worked out how to consistently
that combines graphical and structural information. Parallel
define different scenarios, that are triggered subsequently.
events can implement
Based on the defined scenario state chart, scenario test cases
• initial trigger for scenario paths, e.g. emergency, can by identified by traversing along the path and taking
• trigger for usual phase-internal operations without effect into account the parallel events. An algorithm for automatic
on transitions (e.g. different cabin loads), path identification and subsequent test case generation was
• trigger for failures (e.g. cabin smoke / fan failure) or presented in [6] for scenario state charts in the format that
• combinations. was adopted from [5]. The test cases end at final phase states
Fig. 4. Exemplary tabular transition path for a nominal flight scenario.
Fig. 5. Exemplary tabular state path for an emergency flight scenario.
or when the path loops. They are executable either by suitable of the signal profiles that are applied to the SUT interfaces for
control mechanisms to enable specific paths for the holistic each scenario phase.
state chart or by extracting unbranched scenario paths. The
1) System Under Test (SUT) Interface Definition:
first option was demonstrated in [5].
Before defining the environmental elements, the actual inter-
face of the system under test has to be defined, which should
C. System and Environment Definition Layer be used for the scenario test. This predominantly includes
the declaration of the SUTs’ input and output signals by
The activities described in the section aim to define struc- defining signal name, data-type, initial value for inputs and
tured objects in the environment of the system under test. This units. Moreover, special inputs, such as fault trigger for system
should enable a more straight forward and reusable definition model components, can be defined that may be needed to reach
certain scenarios. To be able to reuse the test scenarios at applicable parallel events and it’s trigger for each scenario
different designed stages, all signals are symbolic variables and the environmental component structure.
and abstract from real transmission mechanisms. If more Based on this, sequences of signal functions and segments are
realistic transmission are needed, e.g. for hardware in the loop now assigned to the signal variables as defined in the previous
(HITL) integration, specific drivers need to be set up outside step. Following the objective of re-usability, a library concept
of the test definition process or manually integrated into the for parameterizable signal functions is used. Two fundamental
SUT model. These drivers could be defined on a target test function types are foreseen:
system, such as dSPACE Scalexio, TechSAT ADS2 or Vector • (Atomic) basic functions, that have a representation for
test hardware, which provides the hardware interface e.g. to the test system the scenario is intended to run on.
send signals as CAN messages to real control modules in a • Higher level functions, that are state chart compounds
HITL test. containing basic functions, states and transitions.
The interface information could be loaded from an interface The atomic function library should be fixed while higher
document file, that may be available from a model-based level functions for component models can be defined indi-
development process. vidually as needed. The library concept makes it easier to
2) Environment Component Definition: set up realistic scenarios by having (signal-based) interface
As a second step, the definition and categorization of environ- component models with parameters such as sensors including
mental elements is carried out. Each previously defined SUT noise, different atmosphere models etc. When assigning the
interface signal can subsequently be assigned to one interface library elements to the scenario the function symbols have
element. Furthermore, it is intended to allow the definition of to be mapped to the scenario signals. The sequence of library
signal groups within environmental elements, if it is desired to elements for each signal is defined by an unbranched transition
define a single maneuver or action in the environment that acts path where a transition is taken after predefined times or
on several signals. One example could be the use of two re- arbitrary expressions. The use of common atomic functions
dundant sensors or protocol-based signal groups. Based on the at lowest level enables the translation of the test scenarios to
given signal classification, parallel compound state processes other test scripting languages (e.g. SCXML [7] in its extended
for every environmental element are inserted into each of the form or proprietary solutions). The assignment of parallel
scenario phases of the state chart structure. Similarly, single calculation models is done in the same fashion as for the
signals or signal groups form parallel processes within these phase signals. An example for the definition of sequential
compound states. In that way, sequences of actions or signal linked basic functions is depicted in Figure 6 for a temperature
profiles could be assigned completely separate for each signal demand signal set from the flight deck during cruise phase.
or signal group of each environmental element within each The functional elements, which are all set functions in the
flight phase. An example is visualized on the left side of Figure example, are defined and parameterized in a tabular form. The
6, which includes the environmental elements Cabin Loads, depicted sequence of linked states, which creates the signal
Environment, Flight Management and OVHD, that contains shown on the right side, can then be automatically generated.
the signal Temp Demand. To complete the signal assignment, phase end conditions need
In the case that it is desired to include single mathematical to be defined to trigger the transition to the next phase.
calculation models that apply continuously during the com- These could be already specified textually during the scenario
plete mission irrespective of any scenario, a parallel process definition steps e.g. cruise time and emergency trigger event or
to the scenario state chart is foreseen. For instance this could can be individually defined according to the assigned profile
be models that calculate the atmospheric pressure based on (e.g. speed≤0). For continuous signals care has to be taken,
a height profile that results from the scenario path. It should that the steadiness is fulfilled when transitioning between the
be mentioned that in principal, these models could also be phases. Useful mechanisms will still need to be worked out.
set up outside of the state chart. The parallel calculation
model process can be structured with the same environment IV. C ONCLUSION AND F UTURE W ORK
component definition as for the phases. However, assigned In this paper, a structured procedure was presented to
signals and signal groups are then excluded from the phases. collectively define flight mission-based scenarios in a state-
The modular setup as described prepares a structured and chart formalism and refine them to create realistic signals for
realistic signal assignment and facilitate re-usability through test stimulation. The aim is the verification and validation of
component libraries. Moreover, it creates context for reporting integrated aircraft systems in an operational context. Due to the
and analysis. mostly generic and modular approach, the complex scenario
definition can partitioned into small manageable parts. Hence,
D. Signal and Component Implementation Layer the method would speed up the process and be reusable at
Having defined the hierarchical structure of scenario phases different stages of the system design. The definition of system
and groups of signals as environmental elements in the pre- scenarios in the first step is as far as possible abstracted from
vious steps, the concrete signal profiles are set up in this last the concrete system implementations and can be set up inde-
step. For each phase state, the information gathered before pendently, early in the process. The approach of hierarchical
includes the phase name, scenarios that include the phase, objects for signal groupings in the further modeling achieves a
Fig. 6. Component-embedded sequence of basic functions (left) and resulting signal (right).
highly reusable structure, which could be adapted easily with in the research project AGILE-VT (contract code: 20X1730J),
regard to design changes. Finally, with the concept of basic which is supported by the Federal Ministry of Economic
atomic signal functions as library elements, independence from Affairs and Energy in the national LuFo V-3 program.
test automation systems can be reached, given that suitable
R EFERENCES
conversion mechanisms exist. However, the concept is yet to
be applied in a complete system test campaign. [1] M.Halle and F.Thielecke, ”Tool Chain for Avionics Design, Develop-
ment, Integration and Test”, 1st Workshop on Avionics Systems and
In future works, it still needs to be investigated, if the Software Engineering (AVIOSE’19), Stuttgart, Germany.
scenario definition based on paths and parallel events is [2] ISO/IEC/IEEE International Standard 29119-4:2015 - Software and
mature enough to handle very large and complex systems or systems engineering – Software testing – Part 4: Test techniques.
[3] Everett, Gerald D.; McLeod, Raymond: ”Software testing. Testing across
if further mechanisms need to be developed. The procedure the entire software development life cycle.” Hoboken, New Jersey: IEEE
was implemented as a prototype in Matlab/Simulink using Press Wiley-Interscience a John Wiley & Sons Inc. Publication, 2007.
mixed hybrid graphical and structured editing, which was also [4] Cem Kaner (2003): ”An Introduction to Scenario Testing.” In: Software
Testing & Quality Engineering (STQE).
presented in this paper. [5] Dipl.-Ing. J.Grymlas (2017): ”Systemautomatisierung für den multifunk-
Another open topic is the automatic evaluation of test runs. As tionalen Betrieb von Brennstoffzellen in Verkehrsflugzeugen”, PHDThe-
pointed out in the presentation of the mission-based scenario sis, Hamburg University of Technology, Shaker Verlag, 2017.
[6] J. Grymlas, and F. Thielecke, ”Virtual Integration and Testing of
approach, the defined scenario state-chart represents only the Multifunctional Fuel Cell Systems in Commercial Aircraft,” SAE Int. J.
stimulation part for the test execution. Therefore the definition Aerosp. 6(2):2013, doi:10.4271/2013-01-2281.
of parallel observer logics for evaluation is a major point to [7] World Wide Web Consortium (W3C). (2015) State chart XML
(SCXML): State machine notation for control abstraction. [Online].
be addressed. The evaluations could include specified system Available: https://www.w3.org/TR/scxml/.
functional or performance requirements. [8] J. Grymlas, et al., ”Scenario-based testing of multifunctional fuel cell
Moreover, open parameter definitions shall enable the system- systems using the virtual integration platform VIPER”, in International
Conference on Fundamentals and Developement of Fuel Cells (FDFC),
atic exploration of specific scenarios. The transfer of resulting Toulouse, 2015.
system states over repetitive scenario runs is also considered [9] Harel, D. et al. (2012): ”Multi-modal scenarios revisited: A net-based
in that context, e.g. to assess wear effects. In that context, the representation.” In: Theoretical Computer Science 429, S. 118-127.
[10] La Panzica Manna, Valerio et al. (2015): ”Synthesizing tests for com-
remodeling of realistic scenarios from operational data and binatorial coverage of modal scenario specifications.”, MODELS 2015.
investigation of parameter variations is could be addressed. [11] Harel, D. and Marelly, R. (2003): ”Playing with time: on the specifica-
The integration of operational recordings of signals from real tion and execution of time-enriched LSCs.”, MASCOTS 2002.
[12] Damm, W. and Harel, D. (2001): ”LSCs: Breathing Life into Message
operations will also be worked out for this purpose. This Sequence Charts.”, In: Formal Methods in System Design 19 (1), S.
should expand the segment-based approach that was already 45-80.
demonstrated in [8]. [13] Autili, M. et al. (2007): ”Graphical scenarios for specifying temporal
properties: an automated approach.”, In: Autom Softw Eng 14 (3), S.
Finally, due to intend for automatic test execution the export 293-340.
to common test automation engines should be investigated. [14] ASAM e. V. (2020): ASAM OpenSCENARIO.
Online: https://www.asam.net/standards/detail/openscenario/
[15] German Aerospace Center, DLR (2020): Pegasus Research Project.
ACKNOWLEDGMENT Online: https://www.pegasusprojekt.de/en/home
[16] Windisch, Andreas (2011): ”Suchbasierter Strukturtest für Simulink
I would like to thank Martin Halle, Hauke Höber and the Modelle”, PHDThesis, Berlin Institute of Technology, 2011.
anonymous reviewers of the AvioSE’20 for valuable comments [17] ETSI (2020): Testing and Test Control Notation Version 3 (TTCN-3).
on the draft version. The presented results are part of the work Online: http://www.ttcn-3.org/